Download files from lists
HI all,
I coded a program that can download file from the lists, but i seem that I run slowly and did not show timer and wait function. pls hlp me out in details. Here are the codes:
Dim i, start1, start2 As Integer
Dim extract1, extract2 As String
'changewww.yoursite.com to a web page and what you want to download.
For i = 0 To total - 1
Dim wr As HttpWebRequest = CType(WebRequest.Create(txtLinks.Lines(i)), HttpWebRequest)
Dim ws As HttpWebResponse = CType(wr.GetResponse(), HttpWebResponse)
Dim str As Stream = ws.GetResponseStream()
Dim inBuf(100000) As Byte
Dim bytesToRead As Integer = CInt(inBuf.Length)
Dim bytesRead As Integer = 0
While bytesToRead > 0
Dim n As Integer = str.Read(inBuf, bytesRead, bytesToRead)
If n = 0 Then
Exit While
End If
bytesRead += n
bytesToRead -= n
End While
'change C:\something.exe to the directory you are going to put the file, and what you are going to name it.
start1 = InStr(1, txtLinks.Lines(i), ".pl")
start2 = InStr(1, txtLinks.Lines(i), ".PDF")
extract1 = Mid(txtLinks.Lines(i), start1 + 4, start2 - start1 - 4)
start1 = InStr(start2, txtLinks.Lines(i), "_")
extract2 = Mid(txtLinks.Lines(i), start2 + 10, start1 - start2 - 10)
My.Computer.FileSystem.CreateDirectory("C:\Downloads\" & extract1)
'Dim fstr As New FileStream("C:\" & extract2 & ".gif", FileMode.OpenOrCreate, FileAccess.Write)
Dim fstr As New FileStream("C:\Downloads\" & extract1 & "\" & extract2 & ".gif", FileMode.OpenOrCreate, FileAccess.Write)
fstr.Write(inBuf, 0, bytesRead)
str.Close()
fstr.Close()
Next
One reason why this is slow is that it is synchronous.
If you want to speed thing up you'll need a different architecture one that takes advantage of the asynch capabilities of web request.
That would be using calls to getrequestbegin and getrequestend.....
This will enable you to do at least multiple files at the same time or in parallel.
ALso it appears that you are doing this in Strings? I don't think you need to do that. You can download what you are downloading in chunks.
Well I can do my best....
You see... asynch is a real challenge for a person new to programming. It's going to involve multiple threads.
I'm not sure what you are running, but I've found the documentation to be pretty good from the Systems.Net people and it looks as if you've been reading some of it.
So let's begin at the beginning.....
Here is the way Asynchronous Programming works
You make a request and the request is made. As oon as the request is made
execution is returned to you (the program) to do what you like. This differs from synchronous communication because your execution thread is suspended until a synchronous thread completes.
Sycnh looks like this:
Request
wait until completed.
Asych looks like this:
Request Request Completed
|
V
resume processing
When an asynch request completes it wakes up in a completion routine.
You have a buffer (and a context identifier) and based on the context identitifer you know what to do with the buffer and you do it.
But what might the Asynch capability give you?
It means you can make multiple requests with multiple buffers. This is why a context identitifier is important. It's a system you've made up for your purposes, identifying a buffer and where it fits in the scheme of things.
SO if you want to get your feet wet......
Modify your code to a very simple asynch system. This will be no faster than synch but you can learn about completion routines. Try this.
IO:
RequestBegin
sleep
RequestEnd
Process Buffer
Call IO
This s the same as synch..... only it uses and end action routine. Once you've done that.
We'll go to step 2. :)
