POST Data with .net Html Agility Pack

I’m using the Html Agility Pack for getting and parsing HTML pages. It offers many possibilities, including XPath selectors.

Today I had a problem with posting data to a webpage. It’s possible to
call the HtmlWeb Load function with a second “POST” parameter, but
it offers no easy way to add values to be posted.

After a while of searching I came across this solution: add this two
functions to the HtmlWeb class. (You can get the source in download
section, too)

public HtmlDocument SubmitFormValues (NameValueCollection fv, string url)
    // Attach a temporary delegate to handle attaching
    // the post back data
    PreRequestHandler handler = delegate(HttpWebRequest request) {
        string payload = this.AssemblePostPayload (fv);
            byte[] buff = Encoding.ASCII.GetBytes (payload.ToCharArray ());
            request.ContentLength = buff.Length;
            request.ContentType = "application/x-www-form-urlencoded";
            System.IO.Stream reqStream = request.GetRequestStream ();
            reqStream.Write (buff, 0, buff.Length);
            return true;
    this.PreRequest += handler;
    HtmlDocument doc = this.Load (url, "POST");
    this.PreRequest -= handler;
    return doc;

private string AssemblePostPayload (NameValueCollection fv)
    StringBuilder sb = new StringBuilder ();
    foreach (String key in fv.AllKeys) {
        sb.Append ("&" + Uri.EscapeDataString(key) + "=" + Uri.EscapeDataString(fv.Get(key)));
    return sb.ToString ().Substring (1);


Now posting is easy. Example:

NameValueCollection postData = new NameValueCollection (1);
postData.Add ("name", "value");
doc = this.hw.SubmitFormValues (postData, url);

(hw is an HtmlWeb object and doc an HtmlDocument object)

If you need to use cookies, too, you need to modify the HtmlWeb a
little more, because there is a little bug. First add a new private

private CookieContainer _ck = new CookieContainer();

Then find this section:

if (_useCookies)
        req.CookieContainer = new CookieContainer();

And change it into:

if (_useCookies)
        req.CookieContainer = this._ck;

Now you just have to activate the usage of cookies in your HtmlWeb

this.hw = new HtmlWeb ();
this.hw.UseCookies = true;

That’s it! Enjoy easy form posting with a full cookie jar (:


Now read this

FreeNas Storage Appliance #1: The components

Die deutsche Version dieses Artikels findest du hier. This is a series of posts about building your own custom FreeNAS storage applicance. Part 1: The components Part 2: Unboxing On my search for a reliable storage solution, I’ve came... Continue →