friendsterTalk - Friendster Forum

friendsterTalk - Friendster Forum

Welcome guest! Please Login or Register.

#1  2008-11-11 17:24:06

creamownedz
 active server protocol
creamownedz's display avatar
» FriendsterTalker
PsychoCoder
Class-S
Location: Ftalk & MCTF
Registered: 2008-03-22
Posts: 115
Last visit: Yesterday
Reputation: 7

Retrieve all hyperlinks [c#]

This is a method I use when screen scraping to retrieve all hyperlinks from the generated HTML from a WebClient call


Instructions: Need a reference to the following Namespaces

using System.Text;
using System.Text.RegularExpressions;
using System.Collections;


Code:

/// <summary>
/// method for extracting all URL's from the data being
/// passed to the method. The data being passed will be all
/// the data from a provided URL
/// </summary>
/// <param name="str"></param>
/// <returns></returns>
public ArrayList ExtractLinks(string str)
{
    try
    {
        //ArrayList to hold all the links
        ArrayList linksList = new ArrayList();

        //regex pattern for searching
        string pattern = "href=\"[a-zA-Z./:&\\d_-]+\"";

        //create a new RegEx object
        Regex reg = new Regex(pattern, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture);

        //put all the matches into a MatchCollection
        MatchCollection matches = reg.Matches(str);

        //loop through all the matches
        foreach (Match match in matches)
        {
            foreach (Group group in match.Groups)
            {
                //now we do some string manipulation to pull the "href=" off the link
                string url = group.Value.Replace("href=\"", "");
                url = url.Substring(0, url.IndexOf("\""));

                //add the URL to the list
                linksList.Add(url);

            }
        }

        //now return the populated ArrayList
        return linksList;
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
        return null;
    }
}

...Giving of reputation are highly appreciating for me...
Computers don't make errors what they do they do on purpose
...computer is my expertise...
....an addicted coder....
....don't ripp all my tutorials....

Search Friendstertalk

Board footer

FriendsterTalk is not affiliated with Friendster.com
Copyright © 2002–2009 PunBB

[ 8 queries - 0.622 second ]

Pay Per Click Ads by pay per click advertising by Kontera

FriendsterTalk.com x

Welcome to FriendsterTalk! You'll need to login in order to fully use all the features and view all the sections of this site.

Please register if you're not yet a member. =)