Joomla Scraper, Grabber For Joomla, Joomla Scraper Can Grab Any Content From Any Website Options

[Ivan Stamenov]

A new custom RSS parser is added to the Joomla 2.5 version of the component. This RSS parser is our work and is much faster and less memory consuming than SimplePie and is suitable for parsing large feeds.




With our custom Feed parser you can insert only selected fields from rss feed in content, so, you can use non-standart feeds like xml files with some products and others that are not parsed by default from simplepie.


There are some settings that need to be set for every feed, though. The parser needs to be told where (in which XML tag) to find the relevant content. The following notation must be used:

tag[.required_property_name1:required_property_value1]...[.wanted_property_name]

E.g.:

Код

<link>http://the.link.we.want/</link>

=> set the link tag setting to: link, "get the content of the link tag"

Код

<link rel='alternate'>http://the.link.we.want/</link>

=> set the link tag setting to: link.rel:alternate, "get the content of the link tag with rel property = alternate"

Код

<link rel='alternate' alt='permalink' href='http://the.link.we.want/' />

=> set the link tag setting to: link.rel:alternate.alt:permalink.href, "get the content of the href property of the link tag with rel property = alternate and alt property = permalink"


New version is already uploaded in our file directory.

[Ivan Stamenov]

Version 1.7.6 of the Scraper is available. Now JomSocial and Kunena 2.0 forums are supported.



When importing into JomSocial, the following settings are available:
- Profile owner;
- Post where: "As status update" or "On wall";
- Allow comments.

When importing into Kunena 2.0, the following settings are available:
- Forum
- Author ID: the user ID of the posts' author
- Thread ID (optionsl): If set, all the articles will be imported as new posts in this thread. If 0, each article will be posted as separate thread.

[Web Design Seo]

ACL and import in Jomsocial and Kunena are only for Joomla 2.5 version. Import of Kunena is developed to work only with Kunena 2.0 and over. May be will work with older Kunena versions also, but ia not tested and we don't guarantee this.

New version of Joomla Scraper is already avalable for purchase.

[Web Design Seo]

Today is released new version for Joomla 1.5 - Joomla Scraper 1.6.4. Only Joomla 1.5 version is updated!

New version add automated SEO functions on next level!

New function is added with integration with component Advanced Tags (also updated today). Until now, component add as tags only one word keywords extracted from title.

Now, when automatic articles are added, tag component count phrases in title and in body of article and add configured from you number of top key phrases as tags. In this way you can have more in long tail keywords and key phrases.

In this way will be updated and version for Joomla 2.5. This update pending in next 3-4 days.

[Ivan Stamenov]

New version 1.7.7 is now here. v.1.7.7 of Joomla Scraper is bug fix release. The fixed bugs are as follows:

1. The RSS parser didn't convert all the result fields in UTF-8;
2. The article tags were not correctly added using the Advanced Tags component.

[SilverOne]

Pictures from all functions are below


Custom Rss parser:

In the picture above, according to example give is it word 'link' we have to fill in the field or the true link (=http://the.link.we.want/)?

This post has been edited by SilverOne: Dec 5 2012, 03:45 PM

[Web Design Seo]

You must fill in there the field.

[SilverOne]

the field

You have not answered to my question, also I repeat it:

In the picture above, according to example given is it the word 'link' we have to fill in the field or the true link we have to fill in the field (=http://the.link.we.want/)?

[Web Design Seo]

You must fill in there the field.

You must open rss feed, than you must open xml code of this rss feed (press ctrl+u in browser) and must say parser: Hey, content from field "link" in feed code must go to field in joomla scraper parser options "link".

Is now ok

[Web Design Seo]

We find bug and Scraper version will be updated in next hours: If "download images" function is switched on, Joomla Scraper don't import images in K2 tab "image".

Today is released fix for this case and scraper are updated to versions:
- 1.7.8 for Joomla 2.5
- 1.6.5 for Joomla 1.5

[Web Design Seo]

v.1.7.9 for Joomla 2.5 and 1.6.6 for Joomla 1.5 are released today. This version is only with bug fixes - for some cases of non-import pictures in K2.

[Web Design Seo]

Today Joomla Scraper is updated with this bug fix in aggregator cron.

[Ivan Stamenov]

Joomla Scraper Version 1.8.0 for Joomla 2.5 is here. It allows images without extension to be imported as well.
It does not implement the latest SimplePie parser yet.

[Ivan Stamenov]

Joomla Scraper Version 1.8.1 for Joomla 2.5 is now available.

It allows the latest SimplePie (ver. 1.3.1 - PHP 5.2) to be used in addition to the obsolete 1.1.2 (PHP 4).
One may choose which SimplePie version to use on a per RSS feed basis.
Important note: As the classes in both SimplePie versions have the same names, though, when using the "Import All" button, the first loaded SimplePie version will serve all the remaining feeds as well (regardless of their SimplePie version), because PHP does not allow a class (with the same name) to be re-declared.

Finally xAjax is gone as well.

If we keep the current pace, Scraper ver. 12574545.0.1 will be coming soon...

[Ivan Stamenov]

For those of you, willing to update their SimplePie version to 1.3.1, please follow these steps:

1. Download the latest SimplePie:  simplepie.131.inc.zip ( 63.48K ) Number of downloads: 11
;
2. Extract the archive and rename the contained file to simplepie.inc;
3. Upload it to /administrator/components/com_aggregator/inc/simplepie/ and overwrite the existing file.

If you are using PHP 5.2+, you are strongly encouraged to do so as this will get you rid of all these SimplePie "Deprecated: ..." warnings.


Can be differences in performance between parser versions!
Scraper version have 3 different parsers - 2 simplepie (old and new) and one custom parser. Different simplepie parsers have different performance also.

By default parser in "Joomla Scraper" is latest simplepie. We don't have checked and measured performance of old and new simplepie - may be some feeds will work better with old version of simplepie.

If you get "deprecated" or other error, use new version of simplepie parser.

[Web Design Seo]

Today is released bug fix for pagination in list with all feeds in component. Are changed many files, we can't post here this fix.

To receive update, please, send us email from email used in your order. These three users that have buy extension yesterday and today are already received update.

[pavelKukov]

Today, 2013-01-29 is released new version 1.8.4 of Aggregator-Scraper for Joomla. New version support some new great features.

1. Now is possible to preview imported items. Screenshot:



If you select one or more feeds and click on button "preview", you will see the result content that will be imported. This function is to test and fine tuning of rss feeds configuration.

2. You can limit number of imported articles. Screenshot:



If you write in field "feed limit" some number, for example 5, only first 5 items from this feed will be imported in joomla database.

[Web Design Seo]

Today is released version 1.8.5. Added one new function: keywords filter.



You have now filter to configure keywords - one or more. Item from feed will be imported only if some of keywords is find in title or in item body. This option allow you to filter results by keyword and to build better thematic websites. You can configure one feed many times with different keyword filters and to import results in suitable category.

[Web Design Seo]

Joomla Scraper v.1.8.6 for Joomla 3.0 is released.

Screenshots from Joomla3 version:

Version for Joomla 3.0 support import only in content and K2!

[pavelKukov]

Now Joomla Scraper for Joomla 1.5 and 2.5 have improved content shuffling algorithm and new options for better control and automated manipulation. The new algorithm is more precise and produced texts are more readable. New options and settings are:

Shuffle sentences positions - Shuffle the positions of the sentences in the article.
Shuffle sentences - Shuffle the compound sentences using the given characters as a delimiter
Delimiters for sentences - Delimiters used for splitting text into sentences
Delimiters for sentence parts - Delimiters used for splitting sentence into parts
Punctuation characters - List of characters considered as punctuation and disallowed be at start or end of sentence
Minimum sentence length - Sentences shorter than n characters will be removed. Use this option for cleaning small errors like sentences only from names. Example Mr. Paul Kalkbrenner will be split to two sentences because of the dot after Mr.
Fragment shuffling - Fragment shuffling if allowed (if sentence or part of it contain more than n words and do not contain delimiters, it will be split in to phrases which will be shuffled randomly)
Shuffling long phrases - Long phrases prevention if fragment shuffling is allowed. If given text fragment is built up with less than required number words will not be affected from long phrase detection algorithm (with other words - if you want to shuffle small sentences you must set number here. )
Protected HTML tags - Comma separated list of HTML tags which to be protected when removing HTML. This gives you opportunity to keep some formatting.

Latest updated versions are: v.1.8.8 for Joomla 2.5 and v.1.6.7 for joomla 1.5.

This post has been edited by Web Design Seo: Feb 25 2013, 11:12 AM