Welcome Guest ( Log In | Register )

 Forum Rules Joomla Scraper support
 
Reply to this topicStart new topic
> Crontab In Scraper, only last rss update
_david
post Jan 23 2013, 03:13 PM
Post #1


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



hello
I'm new with Joomla scraper.

so, I read the topic for crontab.

==> cron with my provider very 15 minutes with "mycron.php".
==> I put a crontab on my own computer every 5 minutes ( 5 * * * * curl -sS www.my_website.com/administrator/components/com_aggregator/cron.aggregator.php >>/dev/null )

And cron on each rss at differents hours.

But there is a little problem :

Cron works only with the last "feed url" of my list (the last ID).

Here is an exemple :

1 - My first feed enabled
Код
https://vimeo.com/channels/godsavethewind/videos/rss

2 - My second feed enabled
Код
http://feeds.feedburner.com/UploadsByWapalatv?format=xml

3 - My Third Feed enabled
Код
http://feeds.feedburner.com/Mrjasonpolakow?format=xml



==> Even if every feeds got the same cron (15 15 ***) ==> Only the third is update

Il I disabled the third ===> Only the second feed update.

I have 30 feeds ==> Only the last feed with status "enabled" is update. (


I tried really differents things, I changed crons in feed crontab, every feeds with differents hours for update, every feeds with same hour etc... It's only the last feed enabled (the one with the higher ID) which is enabled.


All is ok on manual update.

Do you have an idea ?

Thank you
Go to the top of the page
 
+Quote Post
Web Design Seo
post Jan 23 2013, 03:34 PM
Post #2


Web Design Seo
****

Group: Root Admin
Posts: 4,161
Joined: 29-April 09
From: Sofia
Member No.: 1



Try to configure cron on every 2 or 3 minute. Your code (5 * * * *) is wrong.

"5 * * * *" mean that import will work on every day and every hour in 5 minute (in 1:05, in 2:05 and so...)

Code for every third minute must be:
Код
*/3 * * * *


--------------------
Правила на форума | Forum Rules | How to receive support. 3D Web Design: Уеб дизайн, Seo оптимизация, Web Site Extensions, Oscommerce Addons, Wordpress plugins and Joomla Extensions. Изработка на уеб сайтове и оптимизация на сайт за търсачки и Seo услуги.
Go to the top of the page
 
+Quote Post
_david
post Jan 23 2013, 03:45 PM
Post #3


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



QUOTE (Web Design Seo @ Jan 23 2013, 04:34 PM) *
Try to configure cron on every 2 or 3 minute. Your code (5 * * * *) is wrong.

"5 * * * *" mean that import will work on every day and every hour in 5 minute (in 1:05, in 2:05 and so...)

Code for every third minute must be:
CODE
*/3 * * * *


yes sorry.

My cron is correct */5 * * * *

And I confirme that I didnt have any update since yesterday except the last feed as I said.

Go to the top of the page
 
+Quote Post
Web Design Seo
post Jan 23 2013, 04:00 PM
Post #4


Web Design Seo
****

Group: Root Admin
Posts: 4,161
Joined: 29-April 09
From: Sofia
Member No.: 1



Try it on every 2 minute. If not work, problem is inside one of feeds, is not finished. Switch off one by one and you will see what feed stop all list.

Other things:
- Check your memory limit (128 mb and over) and max execution time also (30 seconds min.).
- Don't try to import every feed with all extras switched on (scraper, download images and synonyms replacement are harder for server)


--------------------
Правила на форума | Forum Rules | How to receive support. 3D Web Design: Уеб дизайн, Seo оптимизация, Web Site Extensions, Oscommerce Addons, Wordpress plugins and Joomla Extensions. Изработка на уеб сайтове и оптимизация на сайт за търсачки и Seo услуги.
Go to the top of the page
 
+Quote Post
_david
post Jan 23 2013, 05:55 PM
Post #5


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



I tried different things.
First of all I checked memory (128) and timelimit : 300.

crontab every 3 minutes.

with 3 feeds : ( twice with differents feeds and no 'options' checked : scrapper)

1st feed : 10 18 * * *
2nd feed : 12 18 * * *
3nd feed : 18 18 * * *

At 18h09, 18h12 and 18h15 : no feed update.
At 18h18 : only the 3rd feed.

then I tried with more feeds :

1st feed : 10 18 * * *
2nd feed : 12 18 * * *
3nd feed : 18 18 * * *
4rd feed 15 18 * * *
5 feed : 25 18 * * *

==> No update until 18h27 ==> 5th feed update but not others feeds.

Finaly I enabled all feeds with different hours
Last feed at 18h50 ===> No update for all feeds, except last feed I created after 18h50



Go to the top of the page
 
+Quote Post
_david
post Jan 24 2013, 01:43 PM
Post #6


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



Hello

I tried different things today.

First I uninstall the component, delete xx_aggregator from DB and reinstall all.

3 news feeds with exact same parameters and cron */1 * * * *

Код
http://www.lemonde.fr/rss/une.xml
http://www.lemonde.fr/rss/videos.xml
http://www.lemonde.fr/podcast-tele-zapping.xml


==> Again same result : There is an update only for the last feed.

I tried different cron time for the 3 rss too with no succes...
Go to the top of the page
 
+Quote Post
_david
post Jan 24 2013, 02:03 PM
Post #7


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



Hi Again ohmy.gif


I tried somethig desesperate ...

Uninstall Joomla Scraper and installed demo version aggregator_platinum-1.6.5-bundle-joomla-2.5

Same RSS + 2 new ==> CRONS WORK.

Updates are OK !

So please could you correct Scrapper or downgrade me to Aggregator platinium ?

thank you.
Go to the top of the page
 
+Quote Post
Web Design Seo
post Jan 24 2013, 02:09 PM
Post #8


Web Design Seo
****

Group: Root Admin
Posts: 4,161
Joined: 29-April 09
From: Sofia
Member No.: 1



I say you - this is because of switched ON extras. If you have one feed with many articles inside - 50 or more and scraper is switched on, memory used can be 200 or more mb.

Aggregator Scraper is Aggregator platinium plus some more extras. If you just switch off scraper function in your feeds, will be the same thing.



--------------------
Правила на форума | Forum Rules | How to receive support. 3D Web Design: Уеб дизайн, Seo оптимизация, Web Site Extensions, Oscommerce Addons, Wordpress plugins and Joomla Extensions. Изработка на уеб сайтове и оптимизация на сайт за търсачки и Seo услуги.
Go to the top of the page
 
+Quote Post
_david
post Jan 24 2013, 05:42 PM
Post #9


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



QUOTE (_david @ Jan 23 2013, 06:55 PM) *
I tried different things.
First of all I checked memory (128) and timelimit : 300.

with 3 feeds : ( twice with differents feeds and no 'options' checked : scrapper)


I allready tried with all switch off ( switch off scraper function ) and 3 feeds with max 25 articles / feeds.
If I try manually all is OK, all contents are created.
If I try with cron only the last feed is created.

When I check in mysql, Table xx_aggregator is not update except for the last feed.
exemple

3rd feed created 2013-01-24 13:45:33 cron */4 * * * * lastrun 2013-01-24 18:01:42

all other feeds remain enchanged :cron */1 * * * * lastrun 2013-01-24 15:05:01

"lastrun" is update with agregator platiniu 1.6.5

crons from aggregator_platinum-1.6.5-bundle-joomla-2.5 doesnt work like com_aggregator_scraper-J25-1.8.1

I checked cron.agregator.php and there is differences.

require_once( JPATH_COMPONENT.DS.'inc'.DS.'xajax.inc.php');
require_once( JPATH_COMPONENT.DS.'inc'.DS.'simplepie'.DS.'simplepie.inc');

==> There is no xajax.inc.php for com_aggregator_scraper-J25-1.8.1

sorry I can't say more.
I try to mix version replacing cron.php xajac etc... but it works only with the right controller.php

This post has been edited by _david: Jan 24 2013, 05:43 PM
Go to the top of the page
 
+Quote Post
Web Design Seo
post Jan 25 2013, 07:07 AM
Post #10


Web Design Seo
****

Group: Root Admin
Posts: 4,161
Joined: 29-April 09
From: Sofia
Member No.: 1



About lastrun: lastrun is working only when items are imported. If you have content items with the same titles in joomla, there will be no import (joomla duplicate content protection). And lastrun will be not updated.

To import one feed with the same items many times, use option "use safe alias". To check what is giong on is better to use option Email notification - this will email you number of items imported and process time of every feed.

About xajax: xajax is completely removed from latest 2 scraper versions, there is no such code????

This is cron.aggregator.php code:
Код
<?php
define('_JEXEC', 1);
define('_CRONJOB', 1);
defined('DS') or define('DS', DIRECTORY_SEPARATOR);
define('JPATH_BASE', substr(__FILE__,0,strrpos(__FILE__, DS."components")));
define('JPATH_COMPONENT', JPATH_BASE .DS.'components'.DS.'com_aggregator');
require_once(JPATH_BASE.DS.'includes'.DS.'defines.php');
require_once(JPATH_BASE.DS.'includes'.DS.'framework.php');
$mainframe =& JFactory::getApplication('site');
$mainframe->initialise();
require_once(JPATH_COMPONENT.DS.'controller.php');
require_once(JPATH_COMPONENT.DS.'helpers'.DS.'cron.php');
JRequest::setVar('task','cron','get');
$controller = new AggregatorController();
$controller->importcron();
echo 'Done.';


What version of joomla scraper you use?


--------------------
Правила на форума | Forum Rules | How to receive support. 3D Web Design: Уеб дизайн, Seo оптимизация, Web Site Extensions, Oscommerce Addons, Wordpress plugins and Joomla Extensions. Изработка на уеб сайтове и оптимизация на сайт за търсачки и Seo услуги.
Go to the top of the page
 
+Quote Post
_david
post Jan 25 2013, 07:17 AM
Post #11


Newbie
*

Group: Members
Posts: 9
Joined: 23-January 13
Member No.: 1,540



Цитат(Web Design Seo @ Jan 25 2013, 08:07 AM) *
About lastrun: lastrun is working only when items are imported. If you have content items with the same titles in joomla, there will be no import (joomla duplicate content protection). And lastrun will be not updated.

To import one feed with the same items many times, use option "use safe alias".

About xajax: xajax is completely removed from latest scraper version, there is no such code????

What version of joomla scraper you use?

com_aggregator_scraper-J25-1.8.1.zip
....



I know all you said.


I told you :

I try demo version of Joomla platinium with xajax and crons work perfectly with that.

But the scraper version WITHOUT xajax doesnt work correctly for me.

Now I think that we must go on.
It's evident that we don't understand each other. Perhaps my english language
So refund me please and disabled my licence. Perhaps I have a specific server configuration, but if crons don't do there job i can't continue to use this software.

Best regards

This post has been edited by Web Design Seo: Jan 25 2013, 08:34 AM
Go to the top of the page
 
+Quote Post
Web Design Seo
post Jan 25 2013, 08:36 AM
Post #12


Web Design Seo
****

Group: Root Admin
Posts: 4,161
Joined: 29-April 09
From: Sofia
Member No.: 1



Latest version is sent on your email.

This can be difference in performance between platinum and scraper version:
Scraper version have 3 different parsers - 2 simplepie (old and new) and one custom parser. Different simplepie parsers have different performance.

By default in scraper version is latest simplepie. In platinum work old version of simplepie only. We don't have checked and measured performance of old and new simplepie - may be some feeds will work better with old version of simplepie.


--------------------
Правила на форума | Forum Rules | How to receive support. 3D Web Design: Уеб дизайн, Seo оптимизация, Web Site Extensions, Oscommerce Addons, Wordpress plugins and Joomla Extensions. Изработка на уеб сайтове и оптимизация на сайт за търсачки и Seo услуги.
Go to the top of the page
 
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

Collapse

> Similar Topics

  Topic Replies Topic Starter Views Last Action
No new Scraper Do Not Import .webp Image Extension
5 Juan Vicente Pascual 4,574 16th May 2013 - 06:59 AM
Last post by: Juan Vicente Pascual
No New Posts Scraper Update For Aggregator Platinum
plugin for Aggregator Platinum that add grabber functions
1 Web Design Seo 4,266 13th June 2011 - 11:51 AM
Last post by: Web Design Seo
No New Posts Pinned: Scraper Plugin In Wordpress Aggregator Ultimate
grabber functions in seo tool WP aggregator
0 Web Design Seo 13,763 8th June 2011 - 10:11 AM
Last post by: Web Design Seo


 



RSS Lo-Fi Version Time is now: 22nd October 2019 - 01:09 AM
Clicky Web Analytics