Get Deposit application running against f4 #538

bbpennel · 2016-11-11T16:31:25Z

Gets the deposit pipeline functional to the point of being able to deposit a bag and its deposit record.

Configuration, dependency and context adjustments to get deposit application working against fcrepo4-vagrant
Changes to many deposit related files to get everything working together
Added contentRootObject class to support the base of the content tree
Added Class for initializing a fedora 4 repository with the basics for the CDR

…po4 clients. Disabled references to ACL service for the time being. Resolved some dependency issues with commons-lang. Updated to java 8 in the places missed previously. Redis changed to use local server temporarily while the VM doesn't have an instance.

…gic to PIDs class to handle reserved PIDs, such as the content root collections object

… Updated port for fcrepo4 server in integration tests

…lude any models, and added unit test for it

…atements

…sses can override. Premis event object overrides isUnmodified

…eval of staging location for files to be ingested. Fixed retrieval of manifest locations during deposit record ingest

…est jobs

…d local test config

bbpennel · 2016-11-11T16:34:30Z

deposit/src/main/java/edu/unc/lib/deposit/work/DepositEmailHandler.java

@@ -54,8 +49,8 @@
 	@Autowired
 	private Dataset dataset;

-	@Autowired
-	private FedoraAccessControlService accessControlService;
+	//@Autowired


I should have put a comment in here, but this stuff is basically commented out as a reminder until we have access control worked out

harringj · 2016-11-15T21:10:29Z

deposit/src/main/java/edu/unc/lib/deposit/fcrepo4/IngestContentObjectsJob.java

+			URI stagingUri = new URI(stagingPath);
+			file = new File(stagingUri);
+		} catch (URISyntaxException e) {
+			failJob(e, "Unable to staging URI: {0}", stagingPath);


"Unable to resolve staging URI"

lfarrell · 2016-11-16T15:48:53Z

deposit/src/main/java/edu/unc/lib/deposit/work/DepositMessageHandler.java

- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package edu.unc.lib.deposit.work;


Is this whole file being replaced by Camel messaging?

Yes, most likely, and there wasn't anything to submit requests for indexing to in the fcrepo4 branch at this point

lfarrell · 2016-11-16T15:54:01Z

fcrepo-clients/src/test/java/edu/unc/lib/dl/fcrepo4/RepositoryInitializerIT.java

+import edu.unc.lib.dl.rdf.Cdr;
+import edu.unc.lib.dl.util.URIUtil;
+
+public class RepositoryInitializerIT extends AbstractFedoraIT {


What does the IT stand for?

"Integration Test"

harringj · 2016-11-17T15:25:09Z

deposit/src/test/resources/log4j.properties

@@ -15,7 +15,7 @@
 #

 # Root logger set to DEBUG and send to console for testing


This comment seems inaccurate. The logger is set to INFO, not DEBUG, no?

harringj · 2016-11-17T15:27:06Z

fcrepo-clients/src/main/java/edu/unc/lib/dl/event/FilePremisLogger.java


 			events.add(event);
 		}

+		log.debug("Retrieved {} events from file log for object {}",


Do these {} need an index inside them, like {0}?

no, confusingly it works differently for log4j loggers versus the java string formatting method, which is used by the failJob method.

…e default

harringj · 2016-11-17T15:44:12Z

fcrepo-clients/src/main/java/edu/unc/lib/dl/fcrepo4/ContentRootObject.java

+
+	@Override
+	public ContentContainerObject addMember(ContentObject member) throws ObjectTypeMismatchException {
+		// TODO should only allow admin units, but allow folders for the moment


Is there a JIRA ticket for this, @bbpennel ? What do you think, in general, about us creating a new ticket for each TODO (or related set thereof), and putting a link to the ticket in the TODO comment, as well as a link to the TODO locus in the ticket? I think it could help us track this work better and keep pending enhancements more visible. The code itself would then provide some context on what the ticket is about. @lfarrell , do you have an opinion here, too?

Good idea. Since our jira is internal, i feel more inclined to link from the ticket to the TODO.

Another benefit is that if we have to shuffle tickets around we won't have to change the TODOs. One negative is that it is slightly weird to link to code on an unmerged branch that will go away, I'm not 100% sure if the link will stay live

You raise some good points. Let's continue the discussion offline, maybe at our next sprint planning :-)

sure, i've added a comment referencing this todo to the AdminUnit ticket

harringj · 2016-11-17T15:53:00Z

fcrepo-clients/src/main/java/edu/unc/lib/dl/fcrepo4/PremisEventObject.java

@@ -38,4 +38,13 @@ public int compareTo(PremisEventObject o) {
 			return 0;
 		}
 	}
+
+	/**
+	 * Override to assume that the remote version will not change after creation


I find this comment kind of confusing. Maybe break into two sentences?

harringj · 2016-11-17T15:59:01Z

fcrepo-clients/src/main/java/edu/unc/lib/dl/fcrepo4/RepositoryInitializer.java

+
+		// Don't initialize the object if it is already present.
+		if (objectExists(contentRootUri)) {
+			return contentUri;


Should this return contentRootUri?

yup, it totally should

harringj · 2016-11-17T16:21:42Z

fcrepo-clients/src/test/java/edu/unc/lib/dl/fcrepo4/RepositoryInitializerIT.java

+	}
+
+	/**
+	 * Show that additional initialization calls after the first do no cause


...not cause...

bbpennel added 13 commits November 11, 2016 11:24

Added ContentRootObject for base object in the content tree. Added lo…

49705ab

…gic to PIDs class to handle reserved PIDs, such as the content root collections object

Added repository initializer which sets up base objects for the repo.…

efefb61

… Updated port for fcrepo4 server in integration tests

Added new ingest jobs into pipeline

07950bc

Fixed bug with FilePremisLogger where getEvents results would not inc…

e4229bd

…lude any models, and added unit test for it

Added missing constructors to IngestDepositRecordJob and fixed log st…

0238789

…atements

Forcing fedora base to end with a /, parsing etags

f82b1b1

Added isUnmodified helper method to repository objects so that subcla…

7f480b6

…sses can override. Premis event object overrides isUnmodified

Added missing constructors for file availability job

bd255d7

Added main log4j file for deposit app and turned up test logging level

2715a19

Added logging to ingest content job to indicate progress. Fixed retri…

a00df3e

…eval of staging location for files to be ingested. Fixed retrieval of manifest locations during deposit record ingest

Added staging policy manager to spring context and injecting into ing…

0bae7d8

…est jobs

Updated test staging configuration for new reduced structure and fixe…

4ffacee

…d local test config

bbpennel commented Nov 11, 2016

View reviewed changes

harringj reviewed Nov 15, 2016

View reviewed changes

lfarrell reviewed Nov 16, 2016

View reviewed changes

lfarrell approved these changes Nov 16, 2016

View reviewed changes

Fixed typo in failure message

168a482

harringj reviewed Nov 17, 2016

View reviewed changes

Removed fcrepo3 context file for deposit app and renamed fcrepo4 to b…

905e468

…e default

harringj reviewed Nov 17, 2016

View reviewed changes

Fixed comments and return on content root creation method

be927d3

harringj reviewed Nov 17, 2016

View reviewed changes

Added missing headers for Repo init IT

0aac666

bbpennel force-pushed the f4-ingest-deposit-run branch from 5c43701 to 0aac666 Compare November 17, 2016 16:22

bbpennel merged commit 43bf414 into fcrepo4 Nov 17, 2016

bbpennel deleted the f4-ingest-deposit-run branch November 17, 2016 16:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get Deposit application running against f4 #538

Get Deposit application running against f4 #538

bbpennel commented Nov 11, 2016

bbpennel Nov 11, 2016

harringj Nov 15, 2016

lfarrell Nov 16, 2016

bbpennel Nov 16, 2016

lfarrell Nov 16, 2016

harringj Nov 16, 2016

harringj Nov 17, 2016

harringj Nov 17, 2016

bbpennel Nov 17, 2016

harringj Nov 17, 2016

bbpennel Nov 17, 2016 •

edited

bbpennel Nov 17, 2016

harringj Nov 17, 2016

bbpennel Nov 17, 2016

harringj Nov 17, 2016

bbpennel Nov 17, 2016

harringj Nov 17, 2016

bbpennel Nov 17, 2016

harringj Nov 17, 2016

		@@ -15,7 +15,7 @@
		#

		# Root logger set to DEBUG and send to console for testing

Get Deposit application running against f4 #538

Get Deposit application running against f4 #538

Conversation

bbpennel commented Nov 11, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bbpennel Nov 17, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bbpennel Nov 17, 2016 •

edited