Controlling Instagram-import

[OliverPifferi]

It's great to see that the Instagram import-feature has finally arrived but is there any chance to control the queue once the files are uploaded and (seemingly) being processed?

I was quite daring in the beginning and chose 100 pictures to import (which put them in the queue) and added four and a single one afterwards. After one day, there doesn't seem to be any activity in terms of import progress.

Is there any command I may trigger to check the status or cancel the import process on my own, self-hosted instance?

Thanks for any response and for finally bringing this feature!

Best regards,

Oliver

[madranet]

I've not got anything helpful to add, I'm afraid apart from a "me too".

I've tried to import 3 Instagram accounts. The first one [around 20 posts] imported within a few minutes,

However, the other two [with slightly more posts, but not hugely so] have been showing a similar dialogue to the OP's for about half a day now....

It's really not very helpful UI. With no info given and no progress indicator, I'm not sure whether I should give it a bit longer, or try re-importing and run the risk of duplicates, if the first one is actually doing <something> behind the scenes.

[OliverPifferi]

Don't know what has happened (or when), but the 100 initially imported were gone and I tried to work with 10-picture-chunks instead. Anyway, yesterday evening I checked my crontab and found that the entry for the www-user Pixelfed is running with

• • • • • /usr/bin/php /var/www/pixelfed/artisan schedule:run >> /dev/null 2>&1

was missing.

I added this yesterday and maybe it is come kind of coincidence but today the stacks of 10 pictures each were processed quite fast but tagged as unlisted from the local timeline. Anyway, the amount of pictures in my profile has successfully increased. Fortunately, many pics of IG aren't worth importing but maybe I tried to import too much in the first stage.

Anyway, will continue do track this - thanks for your feedback anyway, there is still some distance to walk to ditch each error but a good start is made, I'd wish I'd know how to control this import thing - or cancel it.

[OliverPifferi]

Also strange: I cannot review the imported pictures afterwards and, seemingly, they are not uploaded to S3 within a certain time but seem to stay local. That's also something I need to get clarified 😊 !

[daSmirnov]

Yeah... I just tried 14 posts as a test, and its been sat there for 30 minutes with 14 still to import.

Is it a scheduled job that runs a couple of times a day?

Is something wrong with my install... Who knows! A bit of feedback would be good.

[madranet]

I should probably point out that my issues with importing are on the main pixelfed.social website. I'm not running my own instance. But since the website directs here, I'm presuming it's OK to chime in, when encountering similar problems.

@daSmirnov: Is something wrong with my install... Who knows! A bit of feedback would be good.

If nothing else, it should reassure those of you who are running local instances that ithere isn't necessrily anything wrong with your install --as Pixelfed HQ is suffering similar hiccups.

[madranet]

As an experiment, I tried going through the import process again, on one of the ones that was stuck [see above]. I re-uploaded the JSON.zip file and got the following:

So, again, I'm calling 'unhelpful UI' on this one. Is the import dialogue completely broken here?... or is it not showing me anything to import because all the posts contained in that JSON.zip file are already queued for import from my earlier attempt? --albeit in a queue that doesn't seem to be moving.

If the latter, then a message saying something like "All available posts are already queued for import" might have helped!

[OliverPifferi]

Yeah... I just tried 14 posts as a test, and its been sat there for 30 minutes with 14 still to import.

Is it a scheduled job that runs a couple of times a day?

Is something wrong with my install... Who knows! A bit of feedback would be good.

I don't think it's related to your install - I managed to import over 50 valuable IG pics in sets of ten and still can't reproduce how the import works once you have uploaded the pics and the queue has filled. Maybe the background jobs (https://docs.pixelfed.org/running-pixelfed/installation/#job-queueing) take care of this, maybe it's also the cronjob (https://docs.pixelfed.org/running-pixelfed/installation/#scheduling-periodic-tasks).

Works for me although I cannot review the imported picture from within the "Import"-section and I see that they are all excluded from the timeline (don' know how to change this though) but a little bit more clearance how the process behind the UI works would be great. Additionally, the number of posts increase not instantly but also after a while...

[Morse6690]

My import has been stuck on the same screen as the original post here for 30 hours or so.
How long should this take to go through?
Is there a backlog?
Is there anything I can do to cancel this and try again?
Server: pixelfed.social

[mitexleo]

Kindly run php artisan app:transform-imports if import process is stuck.

[Morse6690]

Now it's been 5 days stuck in this import screen... I'm not really sure what to do about this as this is their server... Isn't this something that pixelfed.social needs to address?

[smt]

I have 110 photos stuck in importing limbo for several days now (on pixelfed.social), Waiting on a resolution from the admins of that site.

[SteveDinn]

My import on pixelfed.social was finally successfully processed at some point, after waiting in limbo for about 2 weeks. My local instance's queue is still stuck after upgrading to the latest commit as of this morning.

[madranet]

My import on pixelfed.social was finally successfully processed at some point, after waiting in limbo for about 2 weeks

Unfortunately my two imports on pixelfed.social are still languishing in limbo [also two weeks on]

[Morse6690]

Mine went through last night.... 12 out of 107 is better than nothing I guess.

[OliverPifferi]

Works well here now and even the past imports are shown now (were empty in the meantime), properly dated. The latest commit definitely changed a lot - still suffering with #4483 though but this one is done (for me, on my instance). 👍🏻

[SteveDinn]

This finally worked for me after I upgraded to what is currently in the dev branch. I had to run php artisan app:transform-imports as per @neonota 's suggestion, and after that, I had to change the owner of the imported files in the app/public/m/_v2/ folder (to match existing files) because the service did not have permission to read them and I was getting placeholder images instead of the photos.

Seems good now though. The import also did not update my total post count, which doesn't ultimately matter, I suppose.

[madranet]

Sigh! --against my better judgement, I thought I'd revisit this today, after [once again!] becoming frustrated beyond belief with Instagram's moronic filters which kept blocking photos of a mediaeval church interior --presumably because it had skulls decorating it!

Anyway, I decided to give Pixelfed another go and start afresh. So deleted my previous account [still languishing on zero posts, after over a month of supposedly importing!] and start again.

I had 160 posts to import from Instagram. So I queued two imports; one with 100 and the other with 60.

After several hours, I've now got to the following stage:

Hmmm... 73 + 8 used to = 81 when I was at school. So where are the other 79 posts? Vanished without trace. So now, I'mfaced with the fun of going through all my imported posts and comparing them against my Instagram archive to see what's missing. Seriously. It would have been quicker and less error-prone to just copy/paste my entire Instagram account post by post into Pixelfed.

As if that wasn't annoying aenough, the crowning turd in the cesspit:

The Instagram account I imported contained digitised versions of some of my old analogue photos. I'd carefully prepared them all and imported them all in chronological order into Instagram and given the original analogue photo capture date in the description. The JSON exported from Instagram correctly contained all those posts in the correct order I had them in Instagram.

However....

...when imported into Pixelfed, the order is completely messed up. There will be a few photos from 1998, then a hadful from 2003, then some from 2000, then 2004.... etc. etc. It's a complete mess and I'm going to have to delete the whole thing and start again --if I can face the prospect of wasting even more of my life on this seemingly impossible task of moving my posts from one social media site to another!

Seriously Pixelfed devs. How can this import feature be so completely and utterly broken? It's not like it's something that was rushed out the door. People have been asking you for it for about 4 years now and, when you finally do get it off your "roadmap" and into production, it's absolutely useless.

Do you seriously expect peope to leave Instagram for this shambles?

[idoodler]

My import has been stuck on the same screen as the original post here for 30 hours or so.
How long should this take to go through?
Is there a backlog?
Is there anything I can do to cancel this and try again?
Server: pixelfed.social

@Morse6690 did you figure anything out?

[Walsonde]

Kindly run php artisan app:transform-imports if import process is stuck.

unfortunately this doesn't work when characters are encoded wrongly by the IG export (as described in [#4479]). Laravel will throw an error message like SQLSTATE[22021]: Character not in repertoire. I guess I'm stuck until #4479 is fixed, at least.