'Order by expression' can't save to shapefile

[qgib]

Author Name: Loren Amelang (@LorenAmelang)
Original Redmine Issue: 19298
Affected QGIS version: 3.2
Redmine category:processing/qgis

---

Algorithm 'Order by expression' starting…
Input parameters:
{ 'ASCENDING' : True, 'EXPRESSION' : '"ID"', 'INPUT' : 'C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/TestProject/Profiles100-50-25LinesOnly/Profiles100-50-25.shp', 'NULLS_FIRST' : False, 'OUTPUT' : 'C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/TestProject/Profiles100-50-25Sorted/Profiles100-50-25Sorted.shp' }

Could not create layer C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/TestProject/Profiles100-50-25Sorted/Profiles100-50-25Sorted.shp: Creation of data source failed (OGR error: C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/TestProject/Profiles100-50-25Sorted/Profiles100-50-25Sorted.shp is not a directory.)
Execution failed after 0.05 seconds

--> It will not let me select a directory for output, it insists I select a .shp file...

I let it use temp file, and it finished without error, but the "Attribute Table" view of the temp file is still not sorted...

It does say, "Be careful, it might not work as expected with some providers, the order might not be kept every time."

--> I've pretty much concluded the "Attribute Table" rows are randomly ordered unless you intentionally sort them - even if the actual shapefile is required to be in a specific order! Can this be?

[qgib]

Author Name: Nyall Dawson (@nyalldawson)

---

Does "C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/TestProject/Profiles100-50-25Sorted" exist? I suspect that's the problem.

Shapefiles aren't a guaranteed ordered format - writing to the format itself will reorder rows internally.

[qgib]

Author Name: Nyall Dawson (@nyalldawson)

---

• status_id was changed from Open to Feedback

[qgib]

Author Name: Loren Amelang (@LorenAmelang)

---

I just tried this with a fresh start of QGIS, and the same project. If I point to a shapefile that exists, it asks if I want to replace it. Definitely never saw that earlier today (not sure I tried it). If I point to an empty folder and give it a name for the shapefile, it works now! I'm almost positive that's the scenario that failed earlier today. It would not let me select just the folder, I had to open it and type a name for the shapefile, and then it failed.

So are you saying the intention is that it must, or must not, exist?

So this reminds me of [[https://issues.qgis.org/issues/19158]]:

Hey - that just began happening to me today! Except I'm on Windows 10 64-bit:
QGIS version
3.0.3-Girona

For several days I could create shapefiles with no problem. Today I changed the rubberband settings available in Settings menu --> Options --> Digitizing. When I went to create a layer to test the new value, I got that yellow top message and the same un-helpful Messages log. I tried a new project, and restarting QGIS, and it always fails now. Setting the line width back to default doesn't help. Fails in a PostGIS project or a file based project.

And then a few days later it was back to working... I have no clue... Whatever it is must still be happening sometimes in 3.2.

As for shapefile row order, a lot of people on the web believe you can sort it and have it stay sorted, and that it matters. I'm looking at ESRI's spec, which says:

ESRI Shapefile Technical Description J-7855 March 1998

Index Records
The I’th record in the index file stores the offset and content length for the I’th record in the main file.

Doesn't say a word about whether the order of those records is preserved - I guess the Index sort could change along with the main file sort. Lots of people on the web have wrong opinions...

[qgib]

Author Name: Alexander Bruy (@alexbruy)

---

Is original issue solved? Can we close this ticket?

[qgib]

Author Name: Loren Amelang (@LorenAmelang)

---

I just tried it again in 3.2.1, and it worked as expected. As in #26988, I guess the problem was randomly temporary... Like #27214, which keeps cycling on and off... "It seems to save the position for awhile, and then stop saving for awhile. No clue what triggers the changes."

So yes, this one is way down the list, go ahead and close it.
#27214 is a serious aggravation and much more frequent example of the randomness.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Loren Amelang wrote:

I just tried it again in 3.2.1, and it worked as expected. As in #26988, I guess the problem was randomly temporary... Like #27214, which keeps cycling on and off... "It seems to save the position for awhile, and then stop saving for awhile. No clue what triggers the changes."

those are completely unrelated issues.

---

• resolution was changed from to fixed/implemented
• status_id was changed from Feedback to Closed