Results 1 to 12 of 12
  1. #1
    mattheww is offline Member
    Join Date
    Dec 2016
    Posts
    5
    Rep Power
    0

    Default How would I go about making a program that shows data from another web site?

    (Sorry if this isn't placed in the correct subforum)
    Basically,I want to make a program that shows live text from a website. This website has text for something that is showing a number which is automatically updated to show the newest version, and I want to import this text from the site into a program that is showing various numbers which are updated every few seconds. Like what text editors or software makers should I use, and what are some scripts that could help me a bit?

  2. #2
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    20,003
    Rep Power
    33

    Default Re: How would I go about making a program that shows data from another web site?

    Does the website have an API you can call to retrieve the data you are looking for?
    If you don't understand my response, don't ignore it, ask a question.

  3. #3
    mattheww is offline Member
    Join Date
    Dec 2016
    Posts
    5
    Rep Power
    0

    Default Re: How would I go about making a program that shows data from another web site?

    I don't think so, it's retrieving this data from an video game server on a xbox one.

  4. #4
    mattheww is offline Member
    Join Date
    Dec 2016
    Posts
    5
    Rep Power
    0

    Default Re: How would I go about making a program that shows data from another web site?

    I'm not 100% sure what an API is, but I wouldn't think so because it's getting this data from servers on an Xbox ONe.

  5. #5
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    20,003
    Rep Power
    33

    Default Re: How would I go about making a program that shows data from another web site?

    If the site allows users to extract data from its pages, it can provide users with commands that can be sent to the site to get that data.
    If you don't understand my response, don't ignore it, ask a question.

  6. #6
    mattheww is offline Member
    Join Date
    Dec 2016
    Posts
    5
    Rep Power
    0

    Default Re: How would I go about making a program that shows data from another web site?

    How would I check if it allows users to extract data? sorry for the late response, long story as to why but it has to do with my PC.

  7. #7
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,993
    Rep Power
    9

    Default Re: How would I go about making a program that shows data from another web site?

    It should be in their Terms of Service. For example, Bloomberg has this section in their ToS (end of section 3):
    You shall not use or attempt to use any “scraper,” “robot,” “bot,” “spider,” “data mining,” “computer code,” or any other automate device, program, tool, algorithm,
    process or methodology to access, acquire, copy, or monitor any portion of the Service, any data or content found on or accessed through the Service, or any
    other Service information without the prior express written consent of BLP. You may not forge headers or otherwise manipulate identifiers in order to disguise
    the origin of any other content.
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2™ © 2013

  8. #8
    mattheww is offline Member
    Join Date
    Dec 2016
    Posts
    5
    Rep Power
    0

    Default Re: How would I go about making a program that shows data from another web site?

    Would thi sbe it?
    "Curse permits you to link to materials on the Websites for personal, non-commercial purposes only. In addition, Curse may now or in the future provide an "Embeddable Player” feature, which you may incorporate into your own personal, non-commercial websites for use in accessing the materials on the Websites, provided that you include a prominent link back to the Curse websites on the pages containing the Embeddable Player. Curse reserves the right to discontinue any aspect of the Curse Websites at any time."

  9. #9
    JosAH's Avatar
    JosAH is offline Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    14,422
    Blog Entries
    7
    Rep Power
    28

    Default Re: How would I go about making a program that shows data from another web site?

    You could check out the URLConnection class; you supply it a target URL and possibly some parameters and the URLConnection does the rest, i.e. it returns the data supplied by that URL. It is up to you how to process that data.

    kind regards,

    Jos
    Build a wall around Donald Trump; I'll pay for it.

  10. #10
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,993
    Rep Power
    9

    Default Re: How would I go about making a program that shows data from another web site?

    Quote Originally Posted by mattheww View Post
    Would thi sbe it?
    "Curse permits you to link to materials on the Websites for personal, non-commercial purposes only. In addition, Curse may now or in the future provide an "Embeddable Player” feature, which you may incorporate into your own personal, non-commercial websites for use in accessing the materials on the Websites, provided that you include a prominent link back to the Curse websites on the pages containing the Embeddable Player. Curse reserves the right to discontinue any aspect of the Curse Websites at any time."
    Read that carefully: you can link to their website, you can use the embeddable player provided you have a link point back. That's all. This doesn't say anything about scraping.

    From the Curse ToS:
    You may not connect to or use the Service in any way not expressly permitted by these Terms of Service. You agree not to access, or attempt to access, the Service by any means other than through the user interface provided through the Service.
    So that's a no.
    Last edited by SurfMan; 12-21-2016 at 09:35 PM.
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2™ © 2013

  11. #11
    jim829 is offline Senior Member
    Join Date
    Jan 2013
    Location
    Northern Virginia, United States
    Posts
    6,226
    Rep Power
    13

    Default Re: How would I go about making a program that shows data from another web site?

    Quote Originally Posted by SurfMan View Post
    Read that carefully: you can link to their website, you can use the embeddable player provided you have a link point back. That's all. This doesn't say anything about scraping.

    From the Curse ToS:

    "You may not connect to or use the Service in any way not expressly permitted by these Terms of Service. You agree not to access, or attempt to access, the Service by any means other than through the user interface provided through the Service."

    So that's a no.
    Are you accessing this via a browser or a special application provided by the site? If the former, as long as you access the service via the user interface (aka a browser - either a well known one or your homegrown one), then I would not see a problem assuming this is for your own consumption. The quoted ToS also doesn't say that you are explicitly permitted to cache their information or look at their cookies, etc but all browsers do it.

    Regards,
    Jim
    Last edited by jim829; 12-21-2016 at 10:20 PM.
    The JavaTM Tutorials | SSCCE | Java Naming Conventions
    Poor planning on your part does not constitute an emergency on my part

  12. #12
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,993
    Rep Power
    9

    Default Re: How would I go about making a program that shows data from another web site?

    Quote Originally Posted by jim829 View Post
    Are you accessing this via a browser or a special application provided by the site? If the former, as long as you access the service via the user interface (aka a browser - either a well known one or your homegrown one), then I would not see a problem assuming this is for your own consumption. The quoted ToS also doesn't say that you are explicitly permitted to cache their information or look at their cookies, etc but all browsers do it.

    Regards,
    Jim
    I am not a lawyer. Just copy/pasting what I see.

    I doubt Curse et al will give a damn if you start scraping their pages for your little project. Unless your program goes viral and a million people will use that instead of their site. You might have some explaining to do to Curse's lawyer. :)

    When using JSoup, you can also set request headers, so forging a fake Firefox User-Agent is easy. Add the cookies from your browser to JSoup and nobody will ever know what you're doing. Go knock yourself out. However, that doesn't remove the fact that the ToS says no.
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2™ © 2013

Similar Threads

  1. I am created this program.but it shows null pointer exception...can any one help me..
    By vetrivelmanian in forum Threads and Synchronization
    Replies: 3
    Last Post: 07-26-2011, 12:30 PM
  2. https site through ProxySerevr program
    By patel_op in forum Networking
    Replies: 1
    Last Post: 04-28-2011, 02:00 PM
  3. Making Data available to every class in a project.
    By JOHNINALBANY in forum New To Java
    Replies: 5
    Last Post: 12-15-2010, 10:09 AM
  4. Replies: 6
    Last Post: 12-09-2010, 09:44 AM
  5. (Java newbie) retrieving data from a web site
    By idum in forum Networking
    Replies: 4
    Last Post: 10-12-2009, 08:37 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •