A http GET request with cookies and referer won't send out?

Two days I've been debuging, but still can't find why I'm in this situation:
My program needs to login to a website, and then get web page which only logined members can get.
First, I GET a login page for initial cookies (asp sessionId);
Then, I send a POST with proper data (to login) and cookies I got in step 1.
Last, I send a GET with the cookies I got in previous step to fetch the page which I really need.

The first and second step works pretty fine, but the last step failed. It Never send out a http GET message (oh, to be honest, sometimes, it sends out). I used HttpLook to make sure this.

here is the trace message:

Making Http Request: url=http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1, info==null:True.
System.Net Verbose: 0 : [1772] WebRequest::Create(http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1)
System.Net Verbose: 0 : [1772] HttpWebRequest#10722382::HttpWebRequest(http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1#-940297018)
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#10722382::HttpWebRequest()
System.Net Verbose: 0 : [1772] Exiting WebRequest::Create() -> HttpWebRequest#10722382
quiting MakeRequest...
System.Net Verbose: 0 : [1772] HttpWebRequest#10722382::GetResponse()
System.Net Information: 0 : [1772] Associating HttpWebRequest#10722382 with ServicePoint#55634702
System.Net Information: 0 : [1772] Associating Connection#41745883 with HttpWebRequest#10722382
System.Net Information: 0 : [1772] Associating HttpWebRequest#10722382 with ConnectStream#32282393
System.Net Information: 0 : [1772] HttpWebRequest#10722382 - Request: GET /index_shiti_more.asp?action=&c_id=42&s_id=133&page=1 HTTP/1.1

“PickManager.vshost.exe”(?): ?“C:\WINDOWS\assembly\GAC_MSIL\System.resources\2.0.0.0_zh-CHS_b77a5c561934e089\System.resources.dll”,?
System.Net Information: 0 : [1772] ConnectStream#32282393 - ?
{
Accept-Language: zh-cn
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
Host: www.akaoyan.com
Connection: Keep-Alive
}?
System.Net Information: 0 : [1772] Connection#41745883 - ?: Version=1.1,StatusCode=200,StatusDescription=OK?
System.Net Information: 0 : [1772] Connection#41745883 - ?
{
Content-Length: 5999
Content-Type: text/html
Date: Wed, 02 Aug 2006 16:01:51 GMT
Set-Cookie: LET%5Fonline=cookies%5Ftrue=yes; path=/,ASPSESSIONIDQQBTTTRD=MKLDPPCAMEBCKNBANJPGLJNM; path=/
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Cache-control: private
}?
System.Net Information: 0 : [1772] ConnectStream#38333420::ConnectStream(? 5999 ?)
System.Net Information: 0 : [1772] Associating HttpWebRequest#10722382 with ConnectStream#38333420
System.Net Information: 0 : [1772] Associating HttpWebRequest#10722382 with HttpWebResponse#66398263
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#10722382::GetResponse() -> HttpWebResponse#66398263
Cookies fetched.
Making Http Request: url=http://www.akaoyan.com/login.asp?action=login_chk, info==null:False.
System.Net Verbose: 0 : [1772] WebRequest::Create(http://www.akaoyan.com/login.asp?action=login_chk)
System.Net Verbose: 0 : [1772] HttpWebRequest#32289439::HttpWebRequest(http://www.akaoyan.com/login.asp?action=login_chk#1907320142)
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#32289439::HttpWebRequest()
System.Net Verbose: 0 : [1772] Exiting WebRequest::Create() -> HttpWebRequest#32289439
System.Net Verbose: 0 : [1772] HttpWebRequest#32289439::GetRequestStream()
System.Net Information: 0 : [1772] Associating HttpWebRequest#32289439 with ServicePoint#55634702
System.Net Information: 0 : [1772] Associating Connection#38678664 with HttpWebRequest#32289439
System.Net Information: 0 : [1772] Associating HttpWebRequest#32289439 with ConnectStream#16206357
System.Net Information: 0 : [1772] HttpWebRequest#32289439 - Request: POST /login.asp?action=login_chk HTTP/1.1

System.Net Information: 0 : [1772] ConnectStream#16206357 - ?
{
Accept-Language: zh-cn
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
Referer: http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1
Content-Type: application/x-www-form-urlencoded
Host: www.akaoyan.com
Cookie: LET%5Fonline=cookies%5Ftrue=yes; ASPSESSIONIDQQBTTTRD=MKLDPPCAMEBCKNBANJPGLJNM
Content-Length: 65
Expect: 100-continue
}?
System.Net Information: 0 : [1772] Connection#38678664 - ?: Version=1.1,StatusCode=100,StatusDescription=Continue?
System.Net Information: 0 : [1772] Connection#38678664 - ?
{

}?
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#32289439::GetRequestStream() -> ConnectStream#16206357
System.Net Verbose: 0 : [1772] ConnectStream#16206357::Write()
System.Net Verbose: 0 : [1772] Data from ConnectStream#16206357::Write
System.Net Verbose: 0 : [1772] 00000000 : 75 73 65 72 6E 61 6D 65-3D 75 6E 69 62 65 74 74 : username=unibett
System.Net Verbose: 0 : [1772] 00000010 : 65 72 26 70 61 73 73 77-6F 72 64 3D 31 39 38 32 : er&password=1982
System.Net Verbose: 0 : [1772] 00000020 : 30 35 30 35 26 69 6D 67-5F 6C 6F 67 69 6E 2E 78 : 0505&img_login.x
System.Net Verbose: 0 : [1772] 00000030 : 3D 31 34 26 69 6D 67 5F-6C 6F 67 69 6E 2E 79 3D : =14&img_login.y=
System.Net Verbose: 0 : [1772] 00000040 : 34 : 4
System.Net Verbose: 0 : [1772] Exiting ConnectStream#16206357::Write()
System.Net Verbose: 0 : [1772] ConnectStream#16206357::Close()
System.Net Verbose: 0 : [1772] Exiting ConnectStream#16206357::Close()
System.Net Verbose: 0 : [1772] ConnectStream#16206357::Close()
System.Net Verbose: 0 : [1772] Exiting ConnectStream#16206357::Close()
quiting MakeRequest...
System.Net Verbose: 0 : [1772] HttpWebRequest#32289439::GetResponse()
System.Net Information: 0 : [1772] Connection#38678664 - ?: Version=1.1,StatusCode=200,StatusDescription=OK?
System.Net Information: 0 : [1772] Connection#38678664 - ?
{
Content-Length: 3074
Cache-Control: private
Content-Type: text/html
Date: Wed, 02 Aug 2006 16:01:51 GMT
Set-Cookie: LET%5Fonline=login%5Fpassword=11fdcf5dc03e6c0a&login%5Fid=&login%5Fusername=unibetter&time%5Fload=2006%2D8%2D3+0%3A01%3A46&guest%5Fname=%D3%CE%BF%CD50305687&cookies%5Ftrue=yes; path=/
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
}?
System.Net Information: 0 : [1772] ConnectStream#55914028::ConnectStream(? 3074 ?)
System.Net Information: 0 : [1772] Associating HttpWebRequest#32289439 with ConnectStream#55914028
System.Net Information: 0 : [1772] Associating HttpWebRequest#32289439 with HttpWebResponse#55432815
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#32289439::GetResponse() -> HttpWebResponse#55432815
Cookies fetched.
Making Http Request: url=http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1, info==null:False.
System.Net Verbose: 0 : [1772] WebRequest::Create(http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1)
System.Net Verbose: 0 : [1772] HttpWebRequest#31853378::HttpWebRequest(http://www.akaoyan.com/index_shiti_more.asp?action=&c_id=42&s_id=133&page=1#-940297018)
System.Net Verbose: 0 : [1772] Exiting HttpWebRequest#31853378::HttpWebRequest()
System.Net Verbose: 0 : [1772] Exiting WebRequest::Create() -> HttpWebRequest#31853378
quiting MakeRequest...
System.Net Verbose: 0 : [1772] HttpWebRequest#31853378::GetResponse()
System.Net Information: 0 : [1772] Associating HttpWebRequest#31853378 with ServicePoint#55634702
System.Net Information: 0 : [1772] Associating Connection#41745883 with HttpWebRequest#31853378
System.Net Verbose: 0 : [3816] HttpWebRequest#31853378::Abort(?)
System.Net Error: 0 : [3816] Exception in the HttpWebRequest#31853378:: - ?
System.Net Error: 0 : [1772] Exception in the HttpWebRequest#31853378::EndGetResponse - ?


The blue texts are frommy own code, the green ones are fromstep1, brown onesstep2, red onesstep3.
("
?" means "time over").

Here is my simple class to download a web page. I've used it for several times in other projects, and it's fine there.
[code=c#]
using System;
using System.Net;
using System.IO;
using System.Text;
using System.Diagnostics;

namespace Deerchao.Utility
{
public class WebPageGetter
{
private static string defaultEncoding = "gb2312";
private static string defaultLanguage = "zh-cn";

public static string Get(string url, GetPageInfo info)
{
int timesTried = 0;
for(timesTried=0; timesTried<1; timesTried++)
{
//try
//{
timesTried++;
HttpWebRequest req = MakeRequest(url, info);
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
return DecodeData(res);
//}
//catch
//{
// continue;
//}
}
return null;
}

private static string DecodeData(HttpWebResponse w)
{
Trace.Write("Entering DecodeData...\r\n");
string strEncoding = null;
string s = w.Headers["content-type"];
if (s != null)
{
int i = s.IndexOf("charset=");
if (i != -1)
{
strEncoding = s.Substring(i + 8);
}
}
MemoryStream ms = new MemoryStream();
byte[] buffer = new byte[0x400];
Stream rs = w.GetResponseStream();
for (int i = rs.Read(buffer, 0, buffer.Length); i > 0; i = rs.Read(buffer, 0, buffer.Length))
{
ms.Write(buffer, 0, i);
}
rs.Close();
if (strEncoding == null)
{
MemoryStream stream = ms;
stream.Seek((long)0, SeekOrigin.Begin);
StreamReader reader = new StreamReader(stream, Encoding.ASCII);
string all = reader.ReadToEnd();
if (all != null)
{
int i = all.IndexOf("charset=");
int j = -1;
if (i != -1)
{
j = all.IndexOf("\"", i);
if (j != -1)
{
int k = i + 8;
strEncoding = all.Substring(k, (j - k) + 1);
char[] chArray = new char[2] { '>', '"' };
strEncoding = strEncoding.TrimEnd(chArray);
}
}
}
}
Encoding encoding = null;
if (strEncoding == null)
{
encoding = Encoding.GetEncoding(defaultEncoding);
}
else
{
try
{
encoding = Encoding.GetEncoding(strEncoding);
}
catch
{
encoding = Encoding.GetEncoding(defaultEncoding);
}
}
ms.Seek((long)0, SeekOrigin.Begin);
StreamReader reader2 = new StreamReader(ms, encoding);
Trace.Write("Content fetched.\r\n");
return reader2.ReadToEnd();
}

public static string Get(string url)
{
return Get(url, null);
}

public static CookieCollection GetCookies(string url, GetPageInfo info)
{
HttpWebRequest req = MakeRequest(url, info);
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
Trace.Write("Cookies fetched.\r\n");
return res.Cookies;
}

public static HttpWebRequest MakeRequest(string url, GetPageInfo info)
{
Trace.Write(string.Format("Making Http Request: url={0}, info==null:{1}.\r\n", url, info==null));
if (url == null || url == string.Empty)
return null;

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);
req.AllowAutoRedirect = true;
req.CookieContainer = new CookieContainer();
req.Headers.Add("Accept-Language", defaultLanguage);
req.Accept = "*/*";
req.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
req.KeepAlive = true;
if (info != null)
{
if (info.Cookies != null)
{
req.CookieContainer.Add(info.Cookies);
}
if (info.Referer != null && info.Referer != string.Empty)
req.Referer = info.Referer;

if (info.PostData != null && info.PostData != string.Empty)
{
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.AllowWriteStreamBuffering = true;
req.Timeout = 8000;
//write post data
byte[] bt = System.Text.Encoding.ASCII.GetBytes(info.PostData);
req.ContentLength = bt.Length;
Stream rs = req.GetRequestStream();
StreamWriter sw = new StreamWriter(rs, Encoding.GetEncoding("ISO-8859-1"));
sw.Write(info.PostData);
sw.Close();
rs.Close();
}
}
Trace.Write("quiting MakeRequest...\r\n");
return req;
}
}

public class GetPageInfo
{
public CookieCollection Cookies = null;
public string PostData = "";
public string Referer = "";
}
}
[/code].

What could be wrong?

[21576 byte] By [deerchao] at [2007-12-23]

.NET Development

Site Classified