How to analyze import behaviour

[ursbraem]

Hi

after adding page TS to tx_news, I was able to import a bunch of tt_news items into tx_news. But I have (at least) two Issues now I don't know how to debug.

• Nothing happening to categories. sys_category and sys_category_record_mm stay empty. (I only have one category and one subcategory in this example, located on the same page as the news items).
• Not all items are imported. In the list module, I get a different count of the news items, but a diff of the entries rather shows that translations are being lost.

Can you give me some hints to fix this?
Thanks!
Urs

[fsaris]

Hi did you run the category import task?

What version of news are you using? Best to use latest master.

Gr. Frans

[ursbraem]

Hi Frans, thanks for the reply!
I tried with news 3.0.1 (from TER), tt_news 3.6.0 (from TER), news_ttnewsimport 1.0.1 (from here) and TYPO3 6.2.6. Is there anything newer?
Cheers
Urs

[fsaris]

Try latest master of ext:news https://github.com/TYPO3-extensions/news there where some fixes regarding important handling.

Gr. Frans

[ursbraem]

awesome, much better! thanks!

[ursbraem]

Ah. Categories work now. The issue with lost localisations persists in the same way as before.
How can I analyze what's going wrong during import, is there a log or any other debugging helpers? (There is nothing new in the PHP error log)

[frans-beech-it]

Not sure if there is a easy way to debug. I did it my self (during development) the old way. Added some debug output in the data provider class.

For the translated records, what are the sys_language_uid values. 0 or higher or also -1?

I know there where/are some problems with sys_language_uid == -1

[ursbraem]

For the translated records, what are the sys_language_uid values. 0 or higher or also -1?

No, just 0 (de) and 1 (fr)

[fsaris]

Are only records with language 0 imported or some of both?
And in the old installation do all records with language = 1 have a parent record with language= 0?

[ursbraem]

Are only records with language 0 imported or some of both?

Some of both - in fact, most of both! Only in some cases, the 2nd language is not imported.

And in the old installation do all records with language = 1 have a parent record with language= 0?

I checked one of the records where only the language = 0 record was imported, where this is true.

I don't want to abuse your help, but in case it gives more insight, here's the SQL for these two records:

INSERT INTO `tt_news` (`uid`, `pid`, `tstamp`, `crdate`, `cruser_id`, `editlock`, `deleted`, `hidden`, `starttime`, `endtime`, `fe_group`, `title`, `datetime`, `image`, `imagecaption`, `imagealttext`, `imagetitletext`, `related`, `short`, `bodytext`, `author`, `author_email`, `category`, `news_files`, `links`, `type`, `page`, `keywords`, `archivedate`, `ext_url`, `sys_language_uid`, `l18n_parent`, `l18n_diffsource`, `no_auto_pb`, `t3ver_oid`, `t3ver_id`, `t3ver_wsid`, `t3ver_label`, `t3ver_state`, `t3ver_stage`, `t3ver_count`, `t3ver_tstamp`) VALUES
(2938, 6, 1400068705, 1400068504, 5, 0, 0, 0, 0, 0, '0', 'www.besserfahrer.ch. Besserfahrer besuchen Kurse. Und fahren sicherer', 1400068440, 'besserfahrer.png', '', '', '', 0, '', 'Die Kampagne Besserfahrer.ch des VSR (Schweizerischer Verkehrssicherheitsrat) ruft 2014 als Schwerpunkt E-Bike-Lenkende dazu auf, sich in der Fahrsicherheit weiter zu bilden. Nur wer regelmässig Kurse besucht, fährt auch im immer dichter werdenden Strassenverkehr sicher. Denn viele Unfälle können verhindert werden – dank mehr theroretischen Know-how und mehr Praxis. Ein Teil der Kurskosten wird den Kursteilnehmenden eines mit dem Qualitätslabel zertifizierten Fahrsicherheitskurses zurück vergütet.<br /><br />E-Bike-Fahrerinnen und –fahrer analysieren die Gefahren im Kreisverkehr, Lastwagenfahrer lernen, wie sie mit voll beladenem Lastwagen in Notsitationen richtig bremsen, Personenwagenführer lernen das Verhalten auf vereisten Strassen. Und Motorradfahrer üben die korrekte Fahrtechnik auf kurvenreichen Strassen.<br /><br />Mehr auf <link http://www.besserfahrer.ch/ _blank external-link-new-window>www.besserfahrer.ch</link><br /><br />', '', '', 1, '', '', 0, 0, '', 0, '', 0, 0, 0x613a31353a7b733a363a2268696464656e223b4e3b733a353a227469746c65223b4e3b733a353a2273686f7274223b4e3b733a383a22626f647974657874223b4e3b733a383a226461746574696d65223b4e3b733a31363a227379735f6c616e67756167655f756964223b4e3b733a353a22696d616765223b4e3b733a31323a22696d61676563617074696f6e223b4e3b733a31323a22696d616765616c7474657874223b4e3b733a383a2263617465676f7279223b4e3b733a393a22737461727474696d65223b4e3b733a373a22656e6474696d65223b4e3b733a32303a2274785f636f75727365735f63616e63656c6c6564223b4e3b733a31383a2274785f636f75727365735f70726963655f64223b4e3b733a31383a2274785f636f75727365735f70726963655f66223b4e3b7d, 0, 0, 0, 0, '', 0, 0, 0, 0),
(2939, 6, 1400068862, 1400068799, 5, 0, 0, 0, 0, 0, '0', 'Campagne www.conducteur-d-elite.ch : suivre des cours, c’est avoir une conduite plus sûre', 1400068740, '', '', '', '', 0, '', 'La campagne conducteur-d-elite.ch du CSR (Conseil suisse de la sécurité routière) entend encourager tous les usagers de véhicules motorisés ainsi que les cyclistes, qu’ils utilisent un vélo conventionnel ou électrique, à suivre des cours de perfectionnement en matière de sécurité routière. Il est en effet devenu indispensable de prendre régulièrement des cours pour rouler en toute sécurité dans un trafic routier de plus en plus dense. Pour inciter les futurs conducteurs d’élite à franchir le pas, le CSR a décidé de rembourser une partie du prix des cours.<br /><br />Dans le cadre des cours de conduite de sécurité certifiés par le label de qualité du CSR, les utilisateurs de vélos électriques apprennent à analyser les dangers encourus dans les giratoires. Les chauffeurs de camion apprennent pour leur part à freiner de manière optimale dans des situations d’urgence quand ils sont au volant d’un camion lourdement chargé. Quant aux motards, ils ont la possibilité d’acquérir la bonne technique du regard sur route sinueuse. Et enfin, les automobilistes apprennent par exemple à réagir correctement en cas de dérapage sur une route verglacée.<br /><br />En savoir plus : <link http://www.conducteur-d-elite.ch/ _blank external-link-new-window>www.conducteur-d-elite.ch</link><br /><br />', '', '', 1, '', '', 0, 0, '', 0, '', 1, 2938, 0x613a31363a7b733a363a2268696464656e223b733a313a2230223b733a353a227469746c65223b733a36393a227777772e6265737365726661687265722e63682e2042657373657266616872657220626573756368656e204b757273652e20556e642066616872656e207369636865726572223b733a353a2273686f7274223b733a303a22223b733a383a22626f647974657874223b733a3936313a22446965204b616d7061676e65204265737365726661687265722e6368206465732056535220285363687765697a6572697363686572205665726b656872737369636865726865697473726174292072756674203230313420616c732053636877657270756e6b7420452d42696b652d4c656e6b656e64652064617a75206175662c207369636820696e2064657220466168727369636865726865697420776569746572207a752062696c64656e2e204e75722077657220726567656c6dc3a473736967204b7572736520626573756368742c2066c3a4687274206175636820696d20696d6d657220646963687465722077657264656e64656e20537472617373656e7665726b656872207369636865722e2044656e6e207669656c6520556e66c3a46c6c65206bc3b66e6e656e2076657268696e646572742077657264656e20e280932064616e6b206d65687220746865726f72657469736368656e204b6e6f772d686f7720756e64206d656872205072617869732e2045696e205465696c20646572204b7572736b6f7374656e20776972642064656e204b7572737465696c6e65686d656e64656e2065696e6573206d69742064656d205175616c6974c3a474736c6162656c207a6572746966697a69657274656e204661687273696368657268656974736b7572736573207a7572c3bc636b2076657267c3bc7465742e3c6272202f3e3c6272202f3e452d42696b652d466168726572696e6e656e20756e6420e2809366616872657220616e616c7973696572656e2064696520476566616872656e20696d204b726569737665726b6568722c204c617374776167656e666168726572206c65726e656e2c2077696520736965206d697420766f6c6c2062656c6164656e656d204c617374776167656e20696e204e6f747369746174696f6e656e2072696368746967206272656d73656e2c20506572736f6e656e776167656e66c3bc68726572206c65726e656e206461732056657268616c74656e206175662076657265697374656e20537472617373656e2e20556e64204d6f746f7272616466616872657220c3bc62656e20646965206b6f7272656b74652046616872746563686e696b20617566206b757276656e7265696368656e20537472617373656e2e3c6272202f3e3c6272202f3e4d65687220617566203c6c696e6b20687474703a2f2f7777772e6265737365726661687265722e63682f205f626c616e6b2065787465726e616c2d6c696e6b2d6e65772d77696e646f773e7777772e6265737365726661687265722e63683c2f6c696e6b3e3c6272202f3e3c6272202f3e223b733a383a226461746574696d65223b733a31303a2231343030303638343430223b733a31363a227379735f6c616e67756167655f756964223b733a313a2230223b733a353a22696d616765223b4e3b733a31323a22696d61676563617074696f6e223b733a303a22223b733a31323a22696d616765616c7474657874223b733a303a22223b733a383a2263617465676f7279223b4e3b733a393a22737461727474696d65223b733a313a2230223b733a373a22656e6474696d65223b733a313a2230223b733a32303a2274785f636f75727365735f63616e63656c6c6564223b733a313a2230223b733a31383a2274785f636f75727365735f70726963655f64223b733a303a22223b733a31383a2274785f636f75727365735f70726963655f66223b733a303a22223b733a31313a226c31386e5f706172656e74223b733a313a2230223b7d, 0, 0, 0, 0, '', 0, 0, 0, 0);

[fsaris]

Are the images correct imported?

[ursbraem]

yes! (as long as imagemagick is running correctly)

[fsaris]

Will try tot find soms time tomorrow to test the sql you posted

[ursbraem]

Nice! I've removed some custom columns from the SQL to avoid conflicts

[velletti]

Today I had the same Problem:

i had 541 News in 8 language in tt_news table:

some only default lang, som localized, some using -1 to show news in all languages and some exists only in one language and had no parent news.

And the (historical grown) data in tt_news was a real mess:

there are entries with language -1 but have translations and i have entries with default language but with a parent id and at least localized entries with missing news parent!

finaly after import i just had 236 news in tx_news_domain_model_news

Resultion:
i modified NewsImportService.php of original news extension

always store syslangauge uid !

+130: $news->setSysLanguageUid($importItem['sys_language_uid']);

put entry only to persistance stack if it IS localized entry

• 384: if ($importItem['sys_language_uid']) {
• 384: if ($importItem['sys_language_uid'] && $importItem['l10n_parent'] > 0 ) {

Fix Problem if tt_news entry is pointing to a missing parent

change lines from
-422 - 431 if ($parentNews !== NULL) {
$importItem['import_id'] .= '|L:' . $importItem['sys_language_uid'];

        $news = $this->initializeNewsRecord($importItem);

        $this->hydrateNewsRecord($news, $importItem, $importItemOverwrite);

        $news->setSysLanguageUid($importItem['sys_language_uid']);
        $news->setL10nParent($parentNews->getUid());
    }

to
$importItem['import_id'] .= '|L:' . $importItem['sys_language_uid'];

    $news = $this->initializeNewsRecord($importItem);

    $this->hydrateNewsRecord($news, $importItem, $importItemOverwrite);

    $news->setSysLanguageUid($importItem['sys_language_uid']);
    if ($parentNews !== NULL) {

        $news->setL10nParent($parentNews->getUid());
    } else {
        $news->setL10nParent(0);

    }

When i now compare old with new with this query, nothing is missing ..:

select OLD.uid,OLD.title , OLD.sys_language_uid, OLD.l18n_parent, NEW.uid , NEW.sys_language_uid, NEW.l10n_parent
from tt_news as OLD
LEFT JOIN tx_news_domain_model_news NEW
ON NEW.import_id = OLD.uid
where OLD.deleted = 0

questions or want the changed file ? just ask or send me an email: see profile
https://github.com/velletti