Removing the double values?

[Lund01]

I was thinking of my next problem on my work.
I need to merge some files, but I also have this problem with one file:

>Contig1
text here
>Contig1
text here
>Contig2
text here
>Contig2
text here

I actualy only need the first >Contig1.
The second >Contig1 I wanna get rid of.
First >Contig2 I want, and also the second >Contig2 I also wanna get rid of.
And that continues through the file.

I was thinking of making a dictionary for this.
But I don't know how to handle his...
Becose I also have te feeling that a dictionary might not be right what I need...

[Tolls]

Depends how big the file is.
Would storing the keys (ie ">Contig1", ">Contig2") be too much (obviously not scalable)?

If that's OK then stick them in a List as you encounter them, after checking if there's one there already:

readline
String not in my list?
then add to list, and do whatever it is you need to do with the text.
else skip to the next key.

[Lund01]

My biggest file has about 400 >Contigs, and they all are written 2 times.
So 800 in total, of where I need 400.

I will look for that list thing.
Sounds as the right thing I might need.
Thanks.

[Tolls]

That's nothing.
Go the route I suggest above.
You're only talking about a few bytes each, times 400, plus a bit of overhead, so a few k.

[wjncross]

If the information is ordered as you have shown here can you not simply maintain a name/value for the last pair loaded and match against the next pair. If it is different then process it and if same discard. Then just repeat until the end of your file.

Both methods should work but this will save some processing time scanning all values as it's only one check.

[Tolls]

That's true.
For some reason I was presuming they were not necessarily in a convenient order.

[Lund01]

I have made this so far:

 
xD

Only, how do I look if there is already something in the arrayList with a certain name? (Like Contig1?)
I know I need an if loop over there. But if (arrayList.exists()) doesn't work.
I also tried if (arrayList >=1). But that is also not it.

[Tolls]

You didn't look terribly closely at the API for ArrayList did you?
:)

[Lund01]

Nope. :p
I used a Java Programming book.

[Tolls]

The API is your friend...:)

[masijade]

Use a Set and simply add them as you come to them.

[Tolls]

Doh!
:)
(damn character limit!)

[Lund01]

Finished that part.
API is idd usefull. LoL

[Lund01]

Finished with this one. :)