Results 1 to 14 of 14
- 06-12-2010, 12:59 AM #1
Member
- Join Date
- Jun 2010
- Posts
- 5
- Rep Power
- 0
Accessing Data From Websites in Java
I am pretty new to Java, so I'm sorry if this is in the wrong place or is a stupid question. Can I get a simple Java console program to access a website and retrieve data to store as a variable? I want a program that can log onto finance.yahoo.com or some other site and retrieve information about the stock market. Is there a relatively simple way to do this?
An example would be ronpaulgraphs.com. He has a java program that would go to Ron Paul's website and get the donation total every 20 minutes and then graph it.
- 06-12-2010, 01:25 AM #2
Senior Member
- Join Date
- Mar 2010
- Posts
- 952
- Rep Power
- 12
You certainly can do what you're talking about, but there are definite disadvantages to this approach. It's called "scraping" and it ties your app tightly to the current design and layout of the web site you are trying to scrape. If it's stock information you're interested in, you're probably better off using one of the many stock quote web services that are out there. (Note that scraping finance.yahoo.com is very likely a violation of their Terms of Service.)
If you really want to do web site scraping, you'll want to use the Parser class:
Parser (Java Platform SE 6)
If you decide to go with the web service approach, you'll want to read the Web Services tutorial:
The WSIT Tutorial
-Gary-
- 06-12-2010, 03:41 AM #3
- Join Date
- Jul 2007
- Location
- Colombo, Sri Lanka
- Posts
- 11,370
- Blog Entries
- 1
- Rep Power
- 26
- 06-15-2010, 04:28 PM #4
Member
- Join Date
- Jun 2010
- Posts
- 5
- Rep Power
- 0
I'm sorry, but that web service tutorial is a bit above my head. Are there any pre-made java projects in eclipse that I could download and play around with? All I know about java is how to write a "hello world" application and basic stuff with variables. Is there anywhere I could get the code for a very basic java program that retrieves stock quotes?
Also, are there any websites that would let me scrape them? What I really want is not the stock quotes, but the data like what is found here: http://finance.yahoo.com/q/ks?s=AAPL+Key+Statistics . Would I get in trouble for scraping yahoo? I doubt web services would have information that is that detailed. Thanks!Last edited by capitalistpig; 06-15-2010 at 04:37 PM.
- 06-15-2010, 06:17 PM #5
Senior Member
- Join Date
- Mar 2010
- Posts
- 952
- Rep Power
- 12
I sympathize, and I have been where you are, but there really aren't any shortcuts to where you want to go. I understand that web services seem complex and difficult to understand when you haven't used them before, but believe me, they are much simpler than scraping a web site in the long run.
If you're just playing around with the finance.yahoo.com data for your own use, I doubt they'd notice. If you're trying to build a business around it, they'll notice for sure, and it will be easy for them to shut you down. Note that they get their financial data from Capital IQ, which is owned by Standard & Poor's. They're all about making the data available to developers, but they're going to want to be paid for it.
I think you simply need to learn more Java skills before you try to tackle this project. Third-party data integration is never as simple as it looks, and is not for beginners. I'm not trying to discourage you -- quite the opposite, in fact -- but there aren't any shortcuts, so don't feel bad that it takes you so long to learn this stuff. It takes everybody a long time.
-Gary-
- 06-16-2010, 06:49 PM #6
- Join Date
- Jul 2007
- Location
- Colombo, Sri Lanka
- Posts
- 11,370
- Blog Entries
- 1
- Rep Power
- 26
- 06-21-2010, 04:06 PM #7
Member
- Join Date
- Jun 2010
- Posts
- 5
- Rep Power
- 0
Would anyone be willing to build a simple program in eclipse that would scrape AAPL: Key Statistics for Apple Inc. - Yahoo! Finance ? I can't afford much, but I would be willing to spend $20 or so.
If not, I'll just have to learn it.
- 06-21-2010, 04:10 PM #8
Learn it. Nobody will do it for $20.
Math problems? Call 1-800-[(10x)(13i)^2]-[sin(xy)/2.362x]
The Ubiquitous Newbie Tips
- 06-21-2010, 04:24 PM #9
Member
- Join Date
- Jun 2010
- Posts
- 5
- Rep Power
- 0
Damn. That sucks. How about $30?
- 06-21-2010, 04:26 PM #10
My hourly rate is $150, go figure :) You might want to ask on rentacoder.
Math problems? Call 1-800-[(10x)(13i)^2]-[sin(xy)/2.362x]
The Ubiquitous Newbie Tips
- 06-21-2010, 04:55 PM #11"Experience is what you get when you don't get what you want" (Dan Stanford)
"Rise and rise again until lambs become lions" (Robin Hood)
- 06-21-2010, 04:59 PM #12
Senior Member
- Join Date
- May 2010
- Posts
- 436
- Rep Power
- 11
- 06-21-2010, 06:19 PM #13
Member
- Join Date
- Jun 2010
- Posts
- 5
- Rep Power
- 0
Damn, $150/hour ain't bad. That makes me want to learn programming. Sure beats mowing lawns and walking dogs!
Oh well, I guess I'll try to learn it. Where should I start? All I know is basic stuff with variables. The Web Service tutorial someone posted is way too complicated for me. Where should I start and what do I need to read to work up to that Web Service tutorial?
- 06-21-2010, 10:11 PM #14
I'd start by reading Sun's comprehensive tutorials on the basics. The Java™ Tutorials Take e-notes, do some practice problems relevant to the tutorial you're reading, etc. And yea it'll take a while but it's worth the climb. (sadd.. quoting Miley Cyrus)
"Experience is what you get when you don't get what you want" (Dan Stanford)
"Rise and rise again until lambs become lions" (Robin Hood)
Similar Threads
-
JNI accessing non primitive data type
By H_P in forum Advanced JavaReplies: 1Last Post: 04-14-2010, 05:43 AM -
Best Websites to learn Java?
By Meta in forum New To JavaReplies: 21Last Post: 03-18-2010, 09:45 AM -
Accessing Data from a .txt file
By Oasis13 in forum New To JavaReplies: 5Last Post: 02-01-2008, 12:16 AM -
Packaging and accessing data files
By todd in forum Advanced JavaReplies: 1Last Post: 08-01-2007, 12:27 AM
Bookmarks