switch to stack.xml from stack.yaml #128

Closed
tfoote opened this Issue May 23, 2012 · 3 comments

Projects

None yet

2 participants

@tfoote
ROS core stacks member

From Ken:

I'd like to propose (for Groovy) getting rid of stack.yaml. I didn't
propose this for Fuerte as we were already behind schedule.

Reasons:

1) It's not actually YAML (lists are represented as strings that you
run split(',') on)
2) It's not well documented (documentation notes several "possibly
vestigial" bits and is incomplete)
3) It has slightly different key names to represent essentially the
same exact information as stack.xml
4) YAML parses 10x (or more) slower than XML, which is noticeable in
larger installations

If there are concerns about creating unnecessary libraries in Catkin
for parsing stack.xml, below I have a small library and program that
takes a (flat) XML document and converts it into two dictionaries: one
with list values and the other with string values. Attribute values
are collapsed into an expanded key; thus ""
becomes d['depend-stack'].append('foo') in the list value
representation. This small program basically provides the same
representation as stack.yaml does and uses only builtin Python
libraries.

The impact of this change would be updating catkin and catkin-debs,
which I argue is less impact than upgrading the entire legacy
toolchain+documentation to (i) respect stack.yaml (ii) map legacy keys
to updated keys. It is already necessary to audit the code to figure
out proper documentation for stack.yaml; I argue that the conversion
has small marginal cost to doing this audit.

Another update that would be necessary is allowing "rosdep" tags in
stack.xml, or changing the meaning of the "" tag, both of
which already seem useful/necessary in the current migration.

  • Ken

SAMPLE RUN:

kwc@bqv:/wg/stor5_home4/kwc/workspace$ ./domdict.py
~/fuerte/src/ros_comm/stack.xml
{author: Maintained by Ken Conley, depend: null, depend-stack: ros,
description: 'ROS
communications-related packages, including core client libraries
(roscpp, rospy,
roslisp) and graph introspection tools (rostopic, rosnode,
rosservice, rosparam).',
description-brief: ros_comm, license: 'BSD,LGPL', review: null,
review-notes: REP
100, review-status: doc reviewed, stack: "\n ", url:
'http://ros.org/wiki/ros_comm'}

author: [Maintained by Ken Conley]
depend: [null]
depend-stack: [ros]
description: ['ROS communications-related packages, including core
client libraries
(roscpp, rospy, roslisp) and graph introspection tools (rostopic,
rosnode, rosservice,
rosparam).']
description-brief: [ros_comm]
license: ['BSD,LGPL']
review: [null]
review-notes: [REP 100]
review-status: [doc reviewed]
stack: ["\n "]

PROGRAM:

#!/usr/bin/env python
from future import print_function

from xml.etree.ElementTree import ElementTree
from collections import defaultdict

def parse_file(f):
list_d = defaultdict(list)
str_d = {}
for e in ElementTree().parse(f).getiterator():
str_d[e.tag] = e.text
list_d[e.tag].append(e.text)
for sub_key, sub_value in e.items():
str_d["%s-%s"%(e.tag, sub_key)] = sub_value
list_d["%s-%s"%(e.tag, sub_key)].append(sub_value)
return str_d, dict(list_d)

def domdict_main():
from optparse import OptionParser
options, args = OptionParser().parse_args()
if len(args) == 0:
parser.error("please specify an input file")
if len(args) > 1:
parser.error("please specify only one input file")
import yaml
print('\n'.join([yaml.safe_dump(x) for x in parse_file(args[0])]))

if name == 'main':
domdict_main()

@dirk-thomas dirk-thomas was assigned May 29, 2012
@tfoote
ROS core stacks member

Another place we need to fix is the ci jobs: http://hudson.willowgarage.com:8080/job/ci-devel-std_msgs-fuerte-ubuntu-lucid-amd64/20/changes They're manually parsing the yaml too :-(

@tfoote
ROS core stacks member

Also as we discussed bloom also needs to be patched to go from stack.xml -> control files

@dirk-thomas
ROS core stacks member

Has been implemented recently.

@dirk-thomas dirk-thomas closed this Jul 2, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment