How to scrape or download a webpage using C#
By
KTB on
Monday, March 05, 2007
Updated
Friday, April 22, 2016
Viewed
72,451 times. (
3 times today.)
Summary
Here I intend to show you how you can use C# and System.Net.HttpWebRequest to scrape or download a webpage.
Similar code can also be used to post forms which utilize both the get and post form methods by adding a few extra lines of code.
Example: Using HttpWebRequest Object
string url = "http://google.com";
string strResult = "";
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(url);
objResponse = objRequest.GetResponse();
using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
{
strResult = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
// Display results to a webpage
Response.Write(strResult);