java使用jsoup动态变化浏览器,避开网站限制,下载网站内容

public static String getHtmlByJsoup(String url) {
    String html = "";
    Random r = new Random();
    int first = r.nextInt(7) + 55;
    int second = r.nextInt(3200);
    int fouth = r.nextInt(140);
    String agent = MessageFormat.format("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/{0}.0.{1}.{2} Safari/537.36 SE 2.X MetaSr 1.0", first , second, fouth);
    Map<String, String> map = new HashMap<String, String>();
    map.put("time", DateUtil.currentTime());
    try {
        // 爬取的目标网站
        Document doc = Jsoup.connect(url).timeout(5000).data(map).ignoreContentType(true).userAgent(agent).get();
        html = doc.html();
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
    }
    return html;
}

发表评论

电子邮件地址不会被公开。 必填项已用*标注