The inside of a computer is as dumb as hell, but it goes like mad! --Richard Feynman

How to scrape or download a webpage using C#


Here I intend to show you how you can use C# and System.Net.HttpWebRequest to scrape or download a webpage.

Similar code can also be used to post forms which utilize both the get and post form methods by adding a few extra lines of code.

Example: Using HttpWebRequest Object

string url = "";
string strResult = "";
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(url);

objResponse = objRequest.GetResponse();

using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
   strResult = sr.ReadToEnd();
   // Close and clean up the StreamReader

// Display results to a webpage