netedit hangs when loading edge data

[ZoltanBaksa]

Hi,
I use the latest dev version v1_7_0+0223-4752229c79.
I have created random trips with randomTrips.py using the option --weights-output-prefix to be able to visualize the start and end edges. When I try to load the generated .dst file into netedit in Data Mode by clicking on File->Data Elements->Load Data Wlements and selecting the generated dst file, netedit hangs and does not responds.

[namdre]

Looks like the algorithm has a major-inefficiency somewhere. It works fine on a small network but takes very long on a large network. (tested with tests/tools/import/OSM/webWizard/osmimport.tools).

I suspect this is related to computing the color-gradient (something along those lines came up previously).

[namdre]

@ZoltanBaksa As a workaround for only looking at the data, you might as well use sumo-gui.

[namdre]

@zoltan: If a rainbow was created, this implies that a valid data range was found. Look into the input file and find an edge that has a different value from the others. Then locate this edge in sumo gui. Activating "show edge color value" may also help to validate what you see.

[namdre]

@palvarezlopez This is what came out of a quick gprof run:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 21.62      3.44     3.44     3970     0.87     3.46  GNEDataInterval::updateGenericDataIDs()
 20.87      6.76     3.32  7898390     0.00     0.00  GUIGlObjectStorage::registerObject(GUIGlObject*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
 14.83      9.12     2.36  7898390     0.00     0.00  std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, GUIGlObject*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, GUIGlObject*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, GUIGlObject*> > >::erase(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
 11.25     10.91     1.79  7898390     0.00     0.00  GUIGlObjectStorage::remove(unsigned int)
 10.43     12.57     1.66  7898390     0.00     0.00  std::_Rb_tree<unsigned int, std::pair<unsigned int const, GUIGlObject*>, std::_Select1st<std::pair<unsigned int const, GUIGlObject*> >, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, GUIGlObject*> > >::erase(unsigned int const&)
  5.91     13.51     0.94 31521800     0.00     0.00  GNEDataSet::AttributeColors::updateValues(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, double)
  2.26     13.87     0.36  7898390     0.00     0.00  GUIGlObject::createFullName[abi:cxx11]() const
  2.26     14.23     0.36  7898390     0.00     0.00  GUIGlObjectStorage::getAllIDs() const
  1.63     14.49     0.26     7940     0.03     0.19  GNEDataInterval::updateAttributeColors()
  1.32     14.70     0.21  7882435     0.00     0.00  GUIGlObject::setMicrosimID(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
  1.07     14.87     0.17 31685379     0.00     0.00  StringUtils::toDouble(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)

[namdre]

Since my network and data file only have ~4000 edges I suspect that registerObject gets called a bit too often.

[namdre]

also, when loading from a file, AttributeColors::updateValues should only be called after loading is complete.

[palvarezlopez]

@ZoltanBaksa I disabled two functions thats create a bottleneck loading data elements. Can you check if in your loading in NETEDIT works now?

[namdre]

pro:

• the loading is very fast now
  con:
• it crashes when selecting a data attribute in the edgeData frame (probably because the color values are not computed)
• nothing can be selected in the interval bar
• the loading-data dialog still isn't updated (and since loading is so fast it's not needed).

[namdre]

Both functions are responsible you disabled have quadratic effort since every call iterates over all interval elements. It should be sufficient to call this once after loading is complete.

[namdre]

works for me now