Results 1 to 8 of 8
  1. #1
    ribbs2521 is offline Member
    Join Date
    Dec 2008
    Posts
    50
    Rep Power
    0

    Default ImageSearch in Java (find an image on the screen)

    I was wondering if anyone can tell me if there is a way to detect an image on the screen? For example, if I have a logo I am looking for or a smiley, is there a way to search and see if that picture is found on the screen somewhere? I've looked around on the forum but I can't seem to find anything.

  2. #2
    ashton is offline Member
    Join Date
    Jan 2009
    Posts
    31
    Rep Power
    0

    Default Oh well,,

    Exactly, what is it in your mind that you want to happen?
    Click an image and know its coordinates or just click it and then know if an image is existing or not?

  3. #3
    paul pasciak is offline Senior Member
    Join Date
    Jul 2008
    Posts
    125
    Rep Power
    0

    Default

    If you intend to search a single image to find certain content,
    like identify how many birds are in a landscape, or fishes in an
    aquarium, I have produced an application that performs some
    principle steps to accomplish this.

    (Translation: the project works under ideal conditions, one
    being that the image's quality be rated as "cartoon".)

    It's not a trivial task. I'm sure there are several approaches
    a programmer might take. I have never found a resource
    on this subject that is easy to read. (PhD level stuff)

    OR MAYBE you are doing something that only requires you
    to search for an image file amongst a collection? For this
    I would turn to File.classes, and performing directory searches.

    Which are you interested in?
    File snooping?
    Or
    Image processing?

  4. #4
    ribbs2521 is offline Member
    Join Date
    Dec 2008
    Posts
    50
    Rep Power
    0

    Default

    Quote Originally Posted by ashton View Post
    Exactly, what is it in your mind that you want to happen?
    Well, I was weary of saying this because I figured things would get out of hand and people would tell me, that's not how you do it..... etc. Anyway, I use a program called AutoHotkey now but since I'm learning Java in school I would like to switch to Java because it is more widely known, I'm assuming. Anyway, I use it to interface with Oracle. Because Oracle is an applet it does not have Control IDs like button1, button2, edit1, etc. So, what I do is I take a screenshot, I crop the titlebar into a PNG file and then, during my program when I send the mouseclick to the coords to open a window in Oracle Applet I loop searching for that image, when the image appears I know that window opened and I continue on with my task.

    Added Note:
    I am a business analyst who creates programs to improve efficiency in our business. I basically write programs to close out RMAs, open RMAs, issue parts, etc.. so that the Customer Service Reps won't have to do it all by hand. It's faster, more reliable since it's the same every time, and it saves on carpal tunnel.

  5. #5
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    Well, I'm not going to lecture you that this is an absolutely horrible concept, but will instead just point out how you can do it.

    What you need to do is take screenshots at regular intervals, and then correlate your png based image with the screen.

    Load your png image using javax.imageio.ImageIO.read("your file name here"), which returns a BufferedImage. Next, take regular screen shots. A simple loop with a Thread.sleep will do this. To do a screenshot in Java, you use java.awt.Robot.createScreenCapture(...arguments), which also returns a BufferedImage.

    Next, use BufferedImage.getRaster() on each of the images to get a Raster, and then use one of the getPixels() methods on the Raster to access the underlying pixel data.

    Now, you loop over the rows and columns of the screen capture data and see if the pixels match teh pixels in your png based image you have stored. This will only work if the pixels exactly match (which will only be true on the machine you did your screen capture on as long as neither the OS or graphics card are modified - this is one reason this is a horrifically aweful approach).

  6. #6
    ribbs2521 is offline Member
    Join Date
    Dec 2008
    Posts
    50
    Rep Power
    0

    Default

    Quote Originally Posted by toadaly View Post
    Well, I'm not going to lecture you that this is an absolutely horrible concept, but will instead just point out how you can do it.
    If you know of a way to manipulate Oracle in a more of a "behind the scenes" concept that would be awesome!. I have searched for years and no-one seems to know or be willing to tell me how to control the Oracle applet without using mouse commands and searching for the title of the window. I have tried using:

    Java Ferret - Gives me the names of the controls but I don't know how to call them.
    COM - I'm not proficient enough to understand COM
    DLL Calls - Same as COM.

    So, if you know of a way that I can control Oracle in a sense like controlling an application via Control IDs that would be wonderful. In the meantime I just wanted to get it working in Java before getting lectured. Your assistance will get me started so I'm ready for lectures lol. I could start a new thread also since it probably warrants a new topic.

    Quote Originally Posted by toadaly View Post
    This will only work if the pixels exactly match (which will only be true on the machine you did your screen capture on as long as neither the OS or graphics card are modified - this is one reason this is a horrifically aweful approach).
    Just to note on this, I haven't had any issues only because our company computers are supplied by one company, therefore are all exactly the same. Luckily lol.
    Last edited by ribbs2521; 02-03-2009 at 06:04 PM.

  7. #7
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    If I were to approach this problem, my first inclination would be to directly manipulate the database using JDBC. Oracle has an ODBC interface as well as an ODBC/JDBC bridge built in.

    If that isn't possible because you're interacting with a 3rd party web app and have no direct access to the database, then my second inclination would be to contact the vendor and explain that you want to automate the form entries. Surely you're not the only customer that wants to do that, unless you're the only customer. If that's the case, they should be willing to bend over backward to help.

    If they blow you off, my third inclination would be to write a stripped down browser, capture their byte code, and use reflection to reverse engineer it.

  8. #8
    ribbs2521 is offline Member
    Join Date
    Dec 2008
    Posts
    50
    Rep Power
    0

    Default

    OK, thanks I'll look into these and let you know. Just an FYI I am an end user on the Oracle app, the company is the actual customer.

Similar Threads

  1. How to find java version
    By roshithmca in forum New To Java
    Replies: 3
    Last Post: 04-27-2012, 06:10 AM
  2. Replies: 4
    Last Post: 01-08-2009, 10:26 PM
  3. java:34:cannot find symbol
    By MissyMadi in forum New To Java
    Replies: 5
    Last Post: 11-09-2008, 06:53 PM
  4. screen scraper for java applet website
    By sheriek in forum Java Applets
    Replies: 3
    Last Post: 09-25-2008, 09:48 AM
  5. Clear screen in java
    By lenny in forum New To Java
    Replies: 1
    Last Post: 07-30-2007, 09:43 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •