Is it possible to query for all the links in a global configuration¿?

[pablogarijo]

Good morning, I am trying to extract some data from my JTS by using this plugin. I would need to know if there is any way to get all the requirements that have links to artifacts from other components. I am able to get links in local components, but not in a global configuration. My links are contained in requiements management project areas.
Does anybody know if this is possible?
Thanks in advance

[barny]

I assume you mean DOORS Next, not JTS. You can't query an artifact for incoming links - to be more precise you can query but you won't get any results. So you'll have to query the other end for its outgoing links. There are examples of querying for artifacts with and without outgoing Satisfies links in the oslcquery readme. I'd expect if you use a GC config then you'll get results from all contributions to the GC which are in the project you query on, because OSLC Query in DN is project-scoped.

Also you've neglected to say which version and iFix you're using.

[barny]

By just selecting one artifact type I get some results

compound queries like the one you've tried should work but haven't exactly been tested a lot, there might be a problem with merging the results, for example the Satisfies link will have a different URI from each component, which might be the problem - I'll create an issue for that.

not the ones between artifacts from the same component.

I'll create another issue for this. Do you get outgoing links within the component when you use the rm local configuration for that component, rather than the GC config?

[barny]

Issues: #24
and #25

[barny]

FYI if you want to query multiple artifact types without using a compound query (which in your case will make two separate OSLC Query calls) then you should be able to use OSLC Query syntax like e.g. -q "oslc:instanceShape in ['Functional Requirement','Non-Functional Requirement']" - this will make a single OSLC Query call. However thinking about it I'm not sure oslcquery will work correctly for a GC where there are multiple URIs for Functional Requirement (i.e. one for each component contributing to the GC)

[pablogarijo]

When I try to make a call for each component by including the -C in the rest api call, I keep getting the same results from the same component every time. And there are many more links that are not appearing in the results. When I query in the rm local configuration I am able to see all the outgoing links from artifacts inside the component to artifacts inside the component

[barny]

I think I've already fixed this issue in my development code - pypi is likely to be updated in the new ear once I've got the regression tests to pass :-)

[pablogarijo]

Good morning,
Has the links issue been updated? I came across some other doubts, let me list them:

• Is it possible to search for the number of artifacts that have an attribute empty?
• Is it possible to search for other link types? (such as satisfied by architecture element) to see some KPIs from the architecture. Is it possible to search directly for architectural elements?
• Is it possible to get data from test cases with this app?
  Thanks in advance and BR

[barny]

Just updated elmclient on pypi to 0.13.0 - try using that.

if I use -v Satisfies I get AFAICT the expected results - includes outgoing links within the component and between components in the same GC. What I'm not convinced about is how the type-system(s) work across components - that may be a limitation of elmclient that I can't easily fix. At the moment I think I merge types with common names - really this merge should be done using RDF URIs. Also an issue that the multiplicity of links isn't propagated correctly across every entry in the Satisfies column. Still hopefully this version is a little better for your needs.

[barny]

Is it possible to search for the number of artifacts that have an attribute empty?

OSLC Query can't do this but it's a feature of oslcquery just like you used -v Satisfies and the results are post-processed to filter out results with no value for Satisifes, if you use -n Satisfies the post-processed removes results with a value fo Satisfies

Is it possible to search for other link types? (such as satisfied by architecture element) to see some KPIs from the architecture. Is it possible to search directly for architectural elements?

These links are incoming from AM so you won't see them in query results from DN because query results from any app don't show incoming links.

TBH I'm not sure if AM supports OSLC Query yet.

Is it possible to get data from test cases with this app?

Yes, see the oslcquery readme for examples. You'll also be able to see outgoing links in results from ETM/QM

[pablogarijo]

And if an attribute could include multiple values, how should I structure a query to see the number of objects that have that attribute with one of the possible values?¿
Imagine the attribute "Number" could have the values 1, 2, 3 and you can select all the values at once. if i use the query -q (Number=1) and the object has selected the values 1, 2 and 3, I wouldnt get the match for that query.
Is there any way to get the data that I am looking for¿?

[barny]

A few ways ways of doing this, at least with 7.0.2SR1: (Number is an Integer, Priority is an enumeration with values Low, Medium, High.
-q "Number in [1,2,3]"
-q "Number>=""1""^xsd:integer and Number<=""3""^^xsd:integer"
-q "Priority=High"
-q "Priority in [Low,High]"

The ^^xsd:integer tells OSLC Query to treat the value (e.g. "1") as an integer - not sure why it's not needed in the first form

The second form is clearly for numeric types - if using enumerations then use the third form for a single value or the fourth form for any one of the values

IIRC there are examples of datetime comparison in the oslcquery readme.

[pablogarijo]

I meant that the attribute could have more than two options selected. for example, if the priority attribute had Low and medium selected, how do I get the number of objects with Low priority? Because if I use what u suggested in that case, it gives me an error on the query

[barny]

It's the way OSLC Query 3.0 works (i.e. not a DN restriction/limitation) that the query syntax can test for in but not for not in - so e.g. if you want the artifacts with only Low and none of the other values selected you will have to post-filter the results. But there's no problem with querying for Low or High (or both) using e.g. -q "Priority in [Low,High]"

[pablogarijo]

Here you can see what is happening to me:

• I include a picture where we can see the attribute subfunction, that can have multiple values at the same time. For the artifacts in the picture, FEA_CAN_DALM and FEA_CAN_ALM are selected
  

• This is the Rest call I am using to see how many requirements have the attribute subfunction with the FEA_CAN_ALM option selected:

oslcquery -J https://doorsng.grupoantolin.com:9443/ -U -P -p "JEEP KM NA24 AmbLp" -C "JEEP KM NA24 AmbLp - Network Management" -s dcterms:title -F "JEEP KM NA24 AmbLp - Network Management_V2.0" -q (Subfunction in [FEA_CAN_ALM])"

the response that I get is:

usage: oslcquery [-h] [-f SEARCHTERMS] [-n NULL] [-o ORDERBY] [-p PROJECTNAME] [-q QUERY] [-r RESOURCETYPE] [-s SELECT] [-u] [-v VALUE] [-A APPSTRINGS]
[-C COMPONENT] [-D DELAYBETWEENPAGES] [-E GLOBALPROJECT] [-F CONFIGURATION] [-G GLOBALCONFIGURATION] [-H SAVECONFIGS] [-J JAZZURL]
[-L LOGLEVEL] [-M MAXRESULTS] [-N] [-O OUTPUTFILE] [-P PASSWORD] [-S] [-T] [-U USERNAME] [-V] [-W] [-X XMLOUTPUTFILE] [-Y] [-Z PROXYPORT]
[--nresults NRESULTS] [--compareresults COMPARERESULTS] [--pagesize PAGESIZE] [--typesystemreport TYPESYSTEMREPORT]
[--cachedays CACHEDAYS] [-0 SAVECREDS] [-1 READCREDS] [-2 ERASECREDS] [-3 SECRET] [-4]
oslcquery: error: unrecognized arguments: in [FEA_CAN_ALM])

And when trying the query -q (Subfunction=FEA_CAN_ALM)":

oslcquery -J https://doorsng.grupoantolin.com:9443/ -U -P -p "JEEP KM NA24 AmbLp" -C "JEEP KM NA24 AmbLp - Network Management" -s dcterms:title -F "JEEP KM NA24 AmbLp - Network Management_V2.0" -q (Subfunction=FEA_CAN_ALM)"

this is the output:

Exception on retrying request. URL: https://doorsng.grupoantolin.com:9443/rm/cm/baseline/_wcQXQFwxEe246PV_7QL-oQ, 404,
GLC self=<elmclient._rm._RMComponent object at 0x000001CA58809F70> name_or_uri='JEEP KM NA24 AmbLp - Network Management_V2.0'
Loading DN shapes: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 12.97 results/s]
Preparing Query
Identifier s=['https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw']
Traceback (most recent call last):
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 93, in _call_userfunc
return f(children)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient_queryparser.py", line 272, in term
raise Exception(f"Single ref {value} not resolved in context {identifier}")
Exception: Single ref FEA_CAN_ALM not resolved in context https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 143, in do_complex_query
(querysteps, uri_to_name_mapping) = self._parse_oslc_query(querystring)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 820, in _parse_oslc_query
querysteps = xformer.transform(tree)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 130, in transform
return self._transform_tree(tree)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 125, in _transform_tree
children = list(self._transform_children(tree.children))
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 116, in _transform_children
yield self._transform_tree(c)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 126, in _transform_tree
return self._call_userfunc(tree, children)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 97, in _call_userfunc
raise VisitError(tree.data, tree, e)
lark.exceptions.VisitError: Error trying to process rule "term":

Single ref FEA_CAN_ALM not resolved in context https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\Scripts\oslcquery-script.py", line 33, in
sys.exit(load_entry_point('elmclient==0.11.3', 'console_scripts', 'oslcquery')())
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\examples\oslcquery.py", line 648, in main
do_oslc_query(sys.argv[1:])
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\examples\oslcquery.py", line 438, in do_oslc_query
results = queryon.do_complex_query( args.resourcetype, querystring=args.query, searchterms=args.searchterms, select=args.select, isnulls=args.null, isnotnulls=args.value
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 146, in do_complex_query
raise Exception( "Error parsing query" )
Exception: Error parsing query

[barny]

Looks like your quoting using " of the -q query is unbalanced and need not use ( ) - should be more like -q "Subfunction in [FEA_CAN_ALM]"

[pablogarijo]

oslcquery -J https://doorsng.grupoantolin.com:9443/ -U -P -p "JEEP KM NA24 AmbLp" -C "JEEP KM NA24 AmbLp - Network Management" -s dcterms:title -F "JEEP KM NA24 AmbLp - Network Management_V2.0" -q "Subfunction in [FEA_CAN_ALM]"

Exception on retrying request. URL: https://doorsng.grupoantolin.com:9443/rm/cm/baseline/_wcQXQFwxEe246PV_7QL-oQ, 404,
GLC self=<elmclient._rm._RMComponent object at 0x000002152C7C9F70> name_or_uri='JEEP KM NA24 AmbLp - Network Management_V2.0'
Loading DN shapes: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 16.40 results/s]
Preparing Query
Identifier s=['https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw']
Traceback (most recent call last):
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 93, in _call_userfunc
return f(children)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient_queryparser.py", line 259, in term
raise Exception(f"List ref {val} not resolved in context {identifier}")
Exception: List ref FEA_CAN_ALM not resolved in context https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 143, in do_complex_query
(querysteps, uri_to_name_mapping) = self._parse_oslc_query(querystring)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 820, in _parse_oslc_query
querysteps = xformer.transform(tree)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 130, in transform
return self._transform_tree(tree)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 125, in _transform_tree
children = list(self._transform_children(tree.children))
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 116, in _transform_children
yield self._transform_tree(c)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 126, in _transform_tree
return self._call_userfunc(tree, children)
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\lark\visitors.py", line 97, in _call_userfunc
raise VisitError(tree.data, tree, e)
lark.exceptions.VisitError: Error trying to process rule "term":

List ref FEA_CAN_ALM not resolved in context https://doorsng.grupoantolin.com:9443/rm/types/AD_Nmo8nQwSEe2lctQcs4AXBw

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Python39\Scripts\oslcquery-script.py", line 33, in
sys.exit(load_entry_point('elmclient==0.11.3', 'console_scripts', 'oslcquery')())
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\examples\oslcquery.py", line 648, in main
do_oslc_query(sys.argv[1:])
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\examples\oslcquery.py", line 438, in do_oslc_query
results = queryon.do_complex_query( args.resourcetype, querystring=args.query, searchterms=args.searchterms, select=args.select, isnulls=args.null, isnotnulls=args.value
File "C:\Users\pablo.garijo\AppData\Roaming\Python\Python39\site-packages\elmclient\oslcqueryapi.py", line 146, in do_complex_query
raise Exception( "Error parsing query" )
Exception: Error parsing query

[barny]

I get that style of error when the enum name I give isn't actually found - there isn't an invisible space at the start/end of the name in its definition is there?. You could try getting a typesystem report by adding --typesystemreport a.html - this will show you the known enumeration names for the attribute. - if there is a space (which seems to be allowed by 7.0.2SR1) then in the report the the enum name will be single-quoted and show the spaces. But if there is a space at start or end I just tried and it can't be resolved by oslcquery - you should probably clean it up in the enum definition and run oslcquery with -W to get the latest typesystem because otherwise it will be cached.

[barny]

OK to work with spaces in a definition name change line 89 (in version 0.13.0) of _queryparser.py from:
SPACYNAME : /[a-zA-Z0-9_][^']*/
to add a space after the [ :
SPACYNAME : /[ a-zA-Z0-9_][^']*/

Now you can use single quote around a name like ' FEA_CAN_ALM ' and it can match a definition that includes space(s) :-)

[barny]

NOTE I updated the text above so the * shows - markup on here was turning it into italics (I think)

[barny]

Did you update elmclient? The first line of oslcquery output starts with the version number e.g. (at present) Version 0.13.0

[pablogarijo]

Good afternoon,
Sorry for not answering earlier. I have now two possible ways to get my job done.

• I checked the types into my DNG component and, as you were telling me, there was a space before some of the values that the attribute subfunction could adopt. I fixed and removed those spaces, but when I generate the typesystem report, the spaces are still there. How could I update the values so that the Typesystem report appears without spaces before them? That way I could filter directly from the search and get my job done.

• The second way is just filtering for Functional and Non functional requirements. I would get a csv with all the requirements listed in it. The thing is that when I write on the query "-s Subfunction" or any multiple selecion attribute, I only get the first one on the report. F.E if in the attribute Area, SW and HW are selected, I only get SW on the report. Is there any way to get all the values selected there? That way I could process the CSV and get my data from there after making the search.

[pablogarijo]

Good morning,
I came across another doubt. How can I use the -s/-n/-v property with an attribute with spaces (F.E "Planned SW Release Version")?

I am getting the next error:

[pablogarijo]

dcterms:title,Planned SW Release Version,dcterms:identifier,r
^
Expected one of:
* COMMA
* LBRACE
* DOT
* COLON

[pablogarijo]

oslcquery -J https://doorsng.grupoantolin.com:9443/ -U -P -p "" -C "SW_DALU" -s dcterms:title -F "SW_DALU_V3.0" -q "((oslc:instanceShape='SW Non-Functional Requirement') || (oslc:instanceShape='SW Functional Requirement'))" -n "Planned SW Release Version" -O result3.csv

this is my query

[barny]

Sorry hadn't realised I hadn't answered this: the -n value has to be in single-quotes ' to get evaluated as an identifier, and because it contains a space the whole thing has to be surrounded by double quotes "
So the correct usage is -n "'Planned SW Release Version'"
Works with version 0.18.0, probably didn't work with version <0.17.0