Skip to content
This repository has been archived by the owner on Dec 22, 2022. It is now read-only.

Commit

Permalink
Pds api 110: unify the behavior of response building (#80)
Browse files Browse the repository at this point in the history 
* fix all the shortcuts to accept application/pds4+json

* changes to support swagger.yaml changes

* middle of refactor

* back to ResponseEntity<Object>

* rework how responses are built

Basically scrapped what was there and reqbuilt it all using a visitor pattern that will hopefully consolidate changes that fix all endpoints instead of so much cut-n-past. There is still a lot of boilerplate thata may be able to be worked out with more effort.

* most errors are cleaned up

Had to add "accept: application/json to the verify scripts. This moved them formward but have some odd results like the query language changed without updating the test. Other items are still failing and will need to track them down one by one but the problems are no longer in the verify scripts.

Added some missed functinality in the product handler.

Cleaned up some logic in the RequestAndResponseContext that does the bulk of the work for mixing and matching the various application type requests. Added a new error to account for when valid information was given (lidvid) but nothing was found despite that.

* last adjustments

The looking up a lidvid from lid required a touch more fine tuning to verify that it found the whole lidvid matches the given to accound for partial lidvids that are more than a lid but less than a full lidvid.

Had to also clean up a List<String>.toArray() that was causing a segfault. Turning a list to [] syntax is not so easy to understand.

* set these to something workable to everyone but me

* add cvs, cvs+text,kvp+json

The work of making the product and adding them all to the endpoints. THe work of adding a serializer is a touch more complicated.

* fixed up the query

The blank query had to use the preset which got lost. Also put the conversion of the names back in place.

* fix what is returned when fields is blanks

Accidentally made it so only the minimal set was returned if fields are left blank. Added a min/max list of fields where min must be a subset of max and empty is a full set without having to define all that may be in elastic search. This min/max approach fixes the blank fields and adds constraints so that pds+ formats will only  every reeturn the allowable keys and nothing else. These are simple to change in the future as well by modifying just the min/max arrays.

Co-authored-by: Al Niessner <Al.Niessner@xxx.xxx>
  • Loading branch information
@al-niessner
al-niessner and Al Niessner committed Nov 18, 2021
1 parent 42fc71d commit 7a9456a
Show file tree
Hide file tree
Showing 28 changed files with 1,035 additions and 698 deletions.
2 changes: 2 additions & 0 deletions docker/Dockerfile.local
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ COPY LICENSE.md /usr/local/registry-api-service-${VERSION}
COPY NOTICE.txt /usr/local/registry-api-service-${VERSION}
COPY README.md /usr/local/registry-api-service-${VERSION}
COPY pom.xml /usr/local/registry-api-service-${VERSION}
#COPY m2.tgz /tmp
#RUN cd /root && tar -zxf /tmp/m2.tgz
COPY src /usr/local/registry-api-service-${VERSION}/src

# Resources shared with the rest of the world
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ POSSIBILITY OF SUCH DAMAGE.
<dependency>
<groupId>gov.nasa.pds</groupId>
<artifactId>api</artifactId>
<version>0.5.0-SNAPSHOT</version>
<version>pds.api.110</version>
</dependency>

<dependency>
Expand Down
3 changes: 0 additions & 3 deletions src/main/java/gov/nasa/pds/api/engineering/configuration/WebMVCConfig.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,9 @@ public void configureMessageConverters(List<HttpMessageConverter<?>> converters)

WebMVCConfig.log.info("Number of converters available " + Integer.toString(converters.size()));
converters.add(new JsonProductSerializer());

converters.add(new Pds4JsonProductSerializer());
converters.add(new Pds4JsonProductsSerializer());

converters.add(new Pds4XmlProductSerializer());

//converters.add(new XmlProductSerializer()); // Product class, application/xml
//converters.add(new Jaxb2RootElementHttpMessageConverter()); // other classes, application/xml
}
Expand Down
181 changes: 74 additions & 107 deletions src/main/java/gov/nasa/pds/api/engineering/controllers/MyBundlesApiController.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
import gov.nasa.pds.api.engineering.elasticsearch.ElasticSearchRegistrySearchRequestBuilder;
import gov.nasa.pds.api.engineering.elasticsearch.business.LidVidNotFoundException;
import gov.nasa.pds.api.engineering.elasticsearch.business.ProductVersionSelector;
import gov.nasa.pds.model.Products;
import gov.nasa.pds.model.Summary;
import gov.nasa.pds.api.engineering.elasticsearch.business.RequestAndResponseContext;
import gov.nasa.pds.api.engineering.exceptions.ApplicationTypeException;
import gov.nasa.pds.api.engineering.exceptions.NothingFoundException;

import com.fasterxml.jackson.databind.ObjectMapper;
import io.swagger.annotations.*;
Expand All @@ -24,11 +25,8 @@
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;


@javax.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2021-02-16T16:35:42.434-08:00[America/Los_Angeles]")
Expand Down Expand Up @@ -68,7 +66,7 @@ public ResponseEntity<Object> bundleByLidvidAll(
@ApiParam(value = "offset in matching result list, for pagination", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue = "0") Integer start,
@ApiParam(value = "maximum number of matching results returned, for pagination", defaultValue = "10") @Valid @RequestParam(value = "limit", required = false, defaultValue = "10") Integer limit)
{
return getAllProductsResponseEntity(lidvid, start, limit);
return this.getAllProductsResponseEntity(lidvid, start, limit);
}


Expand All @@ -85,7 +83,7 @@ public ResponseEntity<Object> getBundles(
}


public ResponseEntity<Products> collectionsOfABundle(
public ResponseEntity<Object> collectionsOfABundle(
@ApiParam(value = "lidvid or lid", required = true) @PathVariable("identifier") String lidvid,
@ApiParam(value = "offset in matching result list, for pagination", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue = "0") Integer start,
@ApiParam(value = "maximum number of matching results returned, for pagination", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue = "100") Integer limit,
Expand All @@ -97,7 +95,7 @@ public ResponseEntity<Products> collectionsOfABundle(
}


public ResponseEntity<Products> collectionsOfABundleAll(
public ResponseEntity<Object> collectionsOfABundleAll(
@ApiParam(value = "lidvid or lid", required = true) @PathVariable("identifier") String lidvid,
@ApiParam(value = "offset in matching result list, for pagination", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue = "0") Integer start,
@ApiParam(value = "maximum number of matching results returned, for pagination", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue = "100") Integer limit,
Expand All @@ -109,7 +107,7 @@ public ResponseEntity<Products> collectionsOfABundleAll(
}


public ResponseEntity<Products> collectionsOfABundleLatest(
public ResponseEntity<Object> collectionsOfABundleLatest(
@ApiParam(value = "lidvid or lid", required = true) @PathVariable("identifier") String lidvid,
@ApiParam(value = "offset in matching result list, for pagination", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue = "0") Integer start,
@ApiParam(value = "maximum number of matching results returned, for pagination", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue = "100") Integer limit,
Expand All @@ -121,17 +119,14 @@ public ResponseEntity<Products> collectionsOfABundleLatest(
}


private Products getBundleCollections(String lidvid, int start, int limit, List<String> fields,
List<String> sort, boolean onlySummary, ProductVersionSelector versionSelector)
private void getBundleCollections(RequestAndResponseContext context)
throws IOException, LidVidNotFoundException
{
long begin = System.currentTimeMillis();

lidvid = productBO.getLidVidDao().getLatestLidVidByLid(lidvid);
String lidvid = productBO.getLidVidDao().getLatestLidVidByLid(context.getLIDVID());
MyBundlesApiController.log.info("Get bundle's collections. Bundle LIDVID = " + lidvid);

List<String> clidvids = null;
if(versionSelector == ProductVersionSelector.ALL)
if(context.getSelector() == ProductVersionSelector.ALL)
{
clidvids = productBO.getBundleDao().getAllBundleCollectionLidVids(lidvid);
}
Expand All @@ -140,124 +135,101 @@ private Products getBundleCollections(String lidvid, int start, int limit, List<
clidvids = productBO.getBundleDao().getBundleCollectionLidVids(lidvid);
}

Set<String> uniqueProperties = new TreeSet<String>();
Products products = new Products();
Summary summary = new Summary();

if (sort == null) { sort = Arrays.asList(); }

summary.setHits(clidvids.size());
summary.setStart(start);
summary.setLimit(limit);
summary.setSort(sort);
products.setSummary(summary);

int size = clidvids.size();
if (size > 0 && start < size && limit > 0)
if (size > 0 && context.getStart() < size && context.getLimit() > 0)
{
int end = start + limit;
int end = context.getStart() + context.getLimit();
if(end > size) end = size;
List<String> ids = clidvids.subList(start, end);
fillProductsFromLidvids(products, uniqueProperties, ids, fields, onlySummary);
List<String> ids = clidvids.subList(context.getStart(), end);
fillProductsFromLidvids(context, ids, -1);
}
else
{
log.warn("Did not find any collections for bundle lidvid: " + lidvid);
}

summary.setProperties(new ArrayList<String>(uniqueProperties));
summary.setTook((int)(System.currentTimeMillis() - begin));
return products;
}


private ResponseEntity<Products> getBundlesCollectionsEntity(String lidvid, int start, int limit,
private ResponseEntity<Object> getBundlesCollectionsEntity(String lidvid, int start, int limit,
List<String> fields, List<String> sort, boolean onlySummary, ProductVersionSelector versionSelector)
{
String accept = this.request.getHeader("Accept");
MyBundlesApiController.log.info("accept value is " + accept);
if ((accept != null
&& (accept.contains("application/json")
|| accept.contains("text/html")
|| accept.contains("application/xml")
|| accept.contains("*/*")))
|| (accept == null))

try
{
RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this.objectMapper, this.getBaseURL(), lidvid, start, limit, fields, sort, onlySummary, versionSelector, this.presetCriteria, accept);
this.getBundleCollections(context);
return new ResponseEntity<Object>(context.getResponse(), HttpStatus.OK);
}
catch (ApplicationTypeException e)
{
log.error("Application type not implemented", e);
return new ResponseEntity<Object>(HttpStatus.NOT_IMPLEMENTED);
}
catch (IOException e)
{
log.error("Couldn't serialize response for content type " + accept, e);
return new ResponseEntity<Object>(HttpStatus.INTERNAL_SERVER_ERROR);
}
catch (LidVidNotFoundException e)
{
log.warn("Could not find lid(vid) in database: " + lidvid);
return new ResponseEntity<Object>(HttpStatus.NOT_FOUND);
}
catch (NothingFoundException e)
{
try
{
Products products = getBundleCollections(lidvid, start, limit, fields, sort, onlySummary, versionSelector);
return new ResponseEntity<Products>(products, HttpStatus.OK);
}
catch (IOException e)
{
log.error("Couldn't serialize response for content type " + accept, e);
return new ResponseEntity<Products>(HttpStatus.INTERNAL_SERVER_ERROR);
}
catch (LidVidNotFoundException e)
{
log.warn("Could not find lid(vid) in database: " + lidvid);
return new ResponseEntity<Products>(HttpStatus.NOT_FOUND);
}
log.warn("Could not find any matching reference(s) in database.");
return new ResponseEntity<Object>(HttpStatus.NOT_FOUND);
}
else return new ResponseEntity<Products>(HttpStatus.NOT_IMPLEMENTED);
}


@Override
public ResponseEntity<Products> productsOfABundle(String lidvid, @Valid Integer start, @Valid Integer limit,
public ResponseEntity<Object> productsOfABundle(String lidvid, @Valid Integer start, @Valid Integer limit,
@Valid List<String> fields, @Valid List<String> sort, @Valid Boolean onlySummary)
{
String accept = this.request.getHeader("Accept");
MyBundlesApiController.log.info("accept value is " + accept);
if ((accept != null
&& (accept.contains("application/json")
|| accept.contains("text/html")
|| accept.contains("application/xml")
|| accept.contains("*/*")))
|| (accept == null))

try
{
RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this.objectMapper, this.getBaseURL(), lidvid, start, limit, fields, sort, false, this.presetCriteria, accept);
this.getProductChildren(context);
return new ResponseEntity<Object>(context.getResponse(), HttpStatus.OK);
}
catch (ApplicationTypeException e)
{
try
{
Products products = this.getProductChildren(lidvid, start, limit, fields, sort, onlySummary);
return new ResponseEntity<Products>(products, HttpStatus.OK);
}
catch (IOException e)
{
log.error("Couldn't serialize response for content type " + accept, e);
return new ResponseEntity<Products>(HttpStatus.INTERNAL_SERVER_ERROR);
}
catch (LidVidNotFoundException e)
{
log.warn("Could not find lid(vid) in database: " + lidvid);
return new ResponseEntity<Products>(HttpStatus.NOT_FOUND);
}
log.error("Application type not implemented", e);
return new ResponseEntity<Object>(HttpStatus.NOT_IMPLEMENTED);
}
catch (IOException e)
{
log.error("Couldn't serialize response for content type " + accept, e);
return new ResponseEntity<Object>(HttpStatus.INTERNAL_SERVER_ERROR);
}
catch (LidVidNotFoundException e)
{
log.warn("Could not find lid(vid) in database: " + lidvid);
return new ResponseEntity<Object>(HttpStatus.NOT_FOUND);
}
catch (NothingFoundException e)
{
log.warn("Could not find any matching reference(s) in database.");
return new ResponseEntity<Object>(HttpStatus.NOT_FOUND);
}
else return new ResponseEntity<Products>(HttpStatus.NOT_IMPLEMENTED);
}


private Products getProductChildren(String lidvid, int start, int limit, List<String> fields, List<String> sort, boolean onlySummary) throws IOException,LidVidNotFoundException
private void getProductChildren(RequestAndResponseContext context) throws IOException,LidVidNotFoundException
{
long begin = System.currentTimeMillis();
lidvid = this.productBO.getLatestLidVidFromLid(lidvid);
String lidvid = this.productBO.getLatestLidVidFromLid(context.getLIDVID());
MyBundlesApiController.log.info("request bundle lidvid, children of products: " + lidvid);

int iteration=0,wsize=0;
Set<String> uniqueProperties = new TreeSet<String>();
List<String> clidvids = productBO.getBundleDao().getBundleCollectionLidVids(lidvid);
List<String> plidvids = new ArrayList<String>();
List<String> wlidvids = new ArrayList<String>();
Products products = new Products();
Summary summary = new Summary();

if (sort == null) { sort = Arrays.asList(); }

summary.setHits(-1);
summary.setLimit(limit);
summary.setSort(sort);
summary.setStart(start);
summary.setTook(-1);
products.setSummary(summary);

if (0 < clidvids.size())
{
Expand All @@ -275,12 +247,12 @@ private Products getProductChildren(String lidvid, int start, int limit, List<St
@SuppressWarnings("unchecked")
List<String> plids = (List<String>)hit.get("product_lidvid");

if (start <= iteration || start < iteration+plids.size()) { wlidvids.addAll(plids); }
if (context.getStart() <= iteration || context.getStart() < iteration+plids.size()) { wlidvids.addAll(plids); }
else { wsize = plids.size(); }
}

if (start <= iteration || start < iteration+wlidvids.size())
{ plidvids.addAll(wlidvids.subList(start <= iteration ? 0 : start-iteration, wlidvids.size())); }
if (context.getStart() <= iteration || context.getStart() < iteration+wlidvids.size())
{ plidvids.addAll(wlidvids.subList(context.getStart() <= iteration ? 0 : context.getStart()-iteration, wlidvids.size())); }

//if (limit <= plidvids.size()) { break; }
//else { iteration = iteration + wlidvids.size() + wsize; }
Expand All @@ -291,16 +263,11 @@ private Products getProductChildren(String lidvid, int start, int limit, List<St

MyBundlesApiController.log.info("found " + Integer.toString(plidvids.size()) + " products in this bundle");

if (plidvids.size() > 0 && limit > 0)
if (plidvids.size() > 0 && context.getLimit() > 0)
{
this.fillProductsFromLidvids(products, uniqueProperties,
plidvids.subList(0, plidvids.size() < limit ? plidvids.size() : limit), fields, onlySummary);
this.fillProductsFromLidvids(context,
plidvids.subList(0, plidvids.size() < context.getLimit() ? plidvids.size() : context.getLimit()), iteration);
}
else MyBundlesApiController.log.warn ("Did not find any products for bundle lidvid: " + lidvid);

summary.setHits(iteration);
summary.setProperties(new ArrayList<String>(uniqueProperties));
summary.setTook((int)(System.currentTimeMillis() - begin));
return products;
}
}
Loading

0 comments on commit 7a9456a

Please sign in to comment.