Home: Products: Plugins: Gossamer Links Plugins: Paid Plugins: 3.x Version: Spider
Other Details
Plugin Description
The idea behind this plugin is to spider another site for links. It will go through the page you enter (currently only one depth), and grab all the links on that page. You can then select which pages you want to spider. Once you decide, then you continue, and it will go to each of the pages, and grab the requested details (description, title and author).
Its all done by meta-tag recognition, so its not fool proof by any means. I got about a 95% success rate with it accurately grabbing the details. You are then shown a list of all the sites that were spidered. It shows the details that were grabbed, and gives you a direct link to add the form (via admin, with all forms pre-completed). Also, inline checking to see if the URL exists in the directory already.
Now works on LinksSQL 2.x + and Gossamer Links 3.x +
Plugin Requirements
- Links SQL 2.1+
- The Following Perl Modules
- CGI.pm
- LWP::Simple
- LWP::UserAgent
- HTML::LinkExtor
- URI::URL
Features
- Ability to turn on/off the Whois checks... NEW
- 2 different types of category dropdown boxes (1 takes longer to generate, but is better on the eyes ... this can be turned on/off as required in the plugins settings) NEW
- Spiders a site, and grab a list of URL's from it...
- Grabs a sites URL, title, description and author (or allows you to use predefined values)
- Checks if the URL is already in your database, and if it is, reports a prominent error to you, so you know about it...
- Other things, which I'll add once I remember them :p
Upcoming Features
- Language defined by user....
- French/German versions...