How to download protected web page using JAVA

[angeles86]

We have a task, to design a class which can download source of any web page. But when I try to test my code and fetch page like http://anidb.net/perl-bin/animedb.pl?show=main – nothing is working.
Standard code like this fails:

import java.net.*;
import java.io.*;

public class URLReader {
    public static void main(String[] args) throws Exception {
        URL link = new URL("http://www.anidb.net/");
        BufferedReader in = new BufferedReader(
        new InputStreamReader(link.openStream()));

        String inputLine;
        while ((inputLine = in.readLine()) != null)
            System.out.println(inputLine);
        in.close();
    }
}

Here is the result I got: Šwq>²"¦§5´ïÇUº=ôÙö?kŠ}~“bd`?l“Ïçz¢Çêõ>"?j׉R“y}K¸\ Ìc_DLÙªÏ_ –óMm_¼_0”•ö°ËC_aí½sî¤ìÁS ‚>dC0ìs_–y¹ñ±ÏÝÜAø%ÈäÖáæ©A@,4x„Š¶ëɃ?
I have tried everything: cookies, header files but nothing seems to work. If you have some hint for me, I will appreciate it. I’ve been thinking about this problem for to weeks. Thanks.

[angeles86]

It's all about gzip. I should use java.util.zip.GZIPInputStream.
Kind people helped me.

[DarrylBurke]

Cross posted
security - How to download protected web page using Java - Stack Overflow
https://forums.oracle.com/forums/thr...ageID=10591775

db

[Norm]

Also posted at How to download protected web page using JAVA