client.executeMethod(httppost卡住不动
写了个 httpclient post 的爬虫,
header部分全部模仿ie
循环的时候会卡住不动,(对单网站循环)
也不超时也不出错,像是假死了一样,很诡异
貌似问题出在 client.executeMethod(httppost); 这行,
不知道具体哪里的问题,有人遇到过这种情况么
System.setProperty( "org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog" ); //关闭日志 Protocol authhttps = new Protocol("https", new AuthSSLProtocolSocketFactory( new URL("file:my.keystore"), "123456", new URL("file:my.truststore"), "123456"), 443); HttpClient client = new HttpClient(); client.getHttpConnectionManager().getParams().setConnectionTimeout(3000); client.getHttpConnectionManager().getParams().setSoTimeout(3000); client.getHostConfiguration().setHost("target-com", 443, authhttps); /* 只能使用相对路径 */ GetMethod httpget = new GetMethod("/link"); client.executeMethod(httpget); PostMethod httppost=new PostMethod("https-://target-com"); httppost.setRequestHeader("Referer", "https-://target-com"); httppost.setRequestHeader("Accept-Language", "en-US"); httppost.setRequestHeader("Content-Type", "application/x-www-form-urlencoded"); httppost.setRequestHeader("User-Agent", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0)"); //httppost.setRequestHeader("Connection", "Keep-Alive"); //httppost.setRequestHeader("Cache-Control", "no-cache"); httppost.setParameter("id", id); httppost.setParameter("link", link); httppost.setParameter("tb", "e1s1"); httppost.setParameter("_eventId", "submit"); client.executeMethod(httppost); String result = new String(httppost.getResponseBody()); httpget.releaseConnection(); //关闭连接 httppost.releaseConnection(); //关闭连接 }
加上循环,跑着跑着就停住不动了,跟假死一样,内存也不见变化,
上面那2个超时设置都不管用,一直卡住不动,都不会超时
|