Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Added ENCRYPTED_PASSWORD property and functionality #4

Merged
merged 125 commits into from Mar 13, 2012

Conversation

Projects
None yet
4 participants
Contributor

prestonmcgowan commented Feb 16, 2012

I have customers who are unable, for security reasons, to place the password for the XCC session in the property file. Also they do not want the password to show up in process listings (top). I have added the ENCRYPTED_PASSWORD flag property to the recordloader codebase. I have also added the com.marklogic.ps.EncryptPassword module to create the encrypted password files.

I realize having an encrypted password file and key file sitting on the filesystem is not too much better. At least these files could be stored separately and controlled and will not show up in a process listing.

Ignore any "github is forcing me" files or updates.

Mike and others added some commits May 19, 2006

Mike first checked in
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@455 e04f4502-82db-0310-b1af-f799f365da79
31fb393
Mike ready to go, I hope
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@456 e04f4502-82db-0310-b1af-f799f365da79
2b1e5e4
Mike html
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@461 e04f4502-82db-0310-b1af-f799f365da79
68a0bd2
Mike fixed a bug related to record-skipping
fixed a potential bug with @euro locals
fixed a bug in the build config
enhanced checkFile to use xdmp:estimate


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@471 e04f4502-82db-0310-b1af-f799f365da79
e33f817
Mike doc enhancements
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@472 e04f4502-82db-0310-b1af-f799f365da79
5e216f7
Mike fixed a bug with null permissions
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@474 e04f4502-82db-0310-b1af-f799f365da79
ae5d125
Mike fixed a copy-paste typo
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@476 e04f4502-82db-0310-b1af-f799f365da79
0044325
Mike added UTF-8 encoding
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@477 e04f4502-82db-0310-b1af-f799f365da79
67e1a1c
Mike changes to make win32 work correctly
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@478 e04f4502-82db-0310-b1af-f799f365da79
a3606e6
Mike added support for ID_NAME=#AUTO
added support for INPUT_ENCODING
added support for INPUT_MALFORMED_ACTION
removed requirement for UTF-8 environment


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@480 e04f4502-82db-0310-b1af-f799f365da79
e77c084
Mike fixed links, added 1.5 requirement
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@481 e04f4502-82db-0310-b1af-f799f365da79
d9f3d62
Mike quick hack for XML_REPAIR_LEVEL
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@482 e04f4502-82db-0310-b1af-f799f365da79
d18e8c7
Mike new features for sgml ingestion
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@483 e04f4502-82db-0310-b1af-f799f365da79
b08a528
Mike fixed retryable handling
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@484 e04f4502-82db-0310-b1af-f799f365da79
72b44c5
Mike fixed the property files
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@486 e04f4502-82db-0310-b1af-f799f365da79
cab8cff
Mike regularized OUTPUT_COLLECTIONS
allow both ' ' and ',' in OUTPUT_COLLECTIONS

refactoring

use nanoseconds for timing


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@497 e04f4502-82db-0310-b1af-f799f365da79
07e9c81
Mike fixed a bug in zipfile handling
fixed various warnings


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@498 e04f4502-82db-0310-b1af-f799f365da79
a22b42a
Mike WIP: refactoring to use java.util.concurrent; working, no jar update yet
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@509 e04f4502-82db-0310-b1af-f799f365da79
06db8f6
Mike seems to work... docs still need updating
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@510 e04f4502-82db-0310-b1af-f799f365da79
13693eb
Mike added code to copy namespace declarations
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@512 e04f4502-82db-0310-b1af-f799f365da79
3f08e05
Mike added missing license decls
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@513 e04f4502-82db-0310-b1af-f799f365da79
792d00b
Mike added ID_NAME=#FILENAME special-case
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@522 e04f4502-82db-0310-b1af-f799f365da79
655f643
Mike improvements and bug fixes
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@523 e04f4502-82db-0310-b1af-f799f365da79
e1a7454
Mike updated reqs
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@525 e04f4502-82db-0310-b1af-f799f365da79
19917a6
Mike fixed record-skipping bug
fixed placekeys bug


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@530 e04f4502-82db-0310-b1af-f799f365da79
950f1a1
Mike fixed a serious bug with SKIP_EXISTING
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@535 e04f4502-82db-0310-b1af-f799f365da79
033526a
Mike migration to DynamicContentStream model: roughly 70% faster
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@536 e04f4502-82db-0310-b1af-f799f365da79
29e3ab1
Mike fixed problems with this morning's refactoring
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@537 e04f4502-82db-0310-b1af-f799f365da79
9aec49c
Mike fixed a bug in read(), and enabled reads with len gt 1
combined with yesterday's change, this speeds up loading to the point where we spend the bulk of time in file and socket I/O


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@538 e04f4502-82db-0310-b1af-f799f365da79
5c15240
Mike copy namespaces by default
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@539 e04f4502-82db-0310-b1af-f799f365da79
2ef5f35
Mike fix for yesterday's fix
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@540 e04f4502-82db-0310-b1af-f799f365da79
427dffa
Mike removed company name
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@541 e04f4502-82db-0310-b1af-f799f365da79
a6b0ef7
Mike improved error reporting
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@544 e04f4502-82db-0310-b1af-f799f365da79
bb55c6b
Mike updating docs
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@546 e04f4502-82db-0310-b1af-f799f365da79
5f7490c
Mike fixed a couple of logic errors
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@547 e04f4502-82db-0310-b1af-f799f365da79
e705fff
Mike added unit test for CW's latest bug report
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@548 e04f4502-82db-0310-b1af-f799f365da79
f81620c
Mike fixed a bug in stdin handling
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@601 e04f4502-82db-0310-b1af-f799f365da79
2e906df
Mike added missing CDATA handler.
improved exception logging.


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@602 e04f4502-82db-0310-b1af-f799f365da79
e9b6c9f
Mike fix for file-handle leak
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@613 e04f4502-82db-0310-b1af-f799f365da79
63e6e47
Mike more tutorial links
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@614 e04f4502-82db-0310-b1af-f799f365da79
df823ed
Mike monolithic #FILENAME processing
minor bug fixes


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@622 e04f4502-82db-0310-b1af-f799f365da79
670bbf4
Mike workaround for xpp bug 249
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@638 e04f4502-82db-0310-b1af-f799f365da79
2034aef
Mike added new config option: THROTTLE_EVENTS_PER_SECOND
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@680 e04f4502-82db-0310-b1af-f799f365da79
ffe4563
Mike added DOCUMENT_FORMAT configuration property
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@682 e04f4502-82db-0310-b1af-f799f365da79
7d96b58
Mike CONNECTION_STRING can now be comma-separated.
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@713 e04f4502-82db-0310-b1af-f799f365da79
669d78c
Mike Adopted patch from Mark Helmstetter,
adding new properties INPUT_STRIP_PREFIX and INPUT_NORMALIZE_PATHS


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@716 e04f4502-82db-0310-b1af-f799f365da79
413fdd5
Mike Fixed a bug in filename id generation
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@720 e04f4502-82db-0310-b1af-f799f365da79
9c8ce91
Mike fixed a typo
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@727 e04f4502-82db-0310-b1af-f799f365da79
ec7ea2d
Mike refactoring and cleanup
check for correct version of XPP3
default behavior is now nicer
* auto-ids for stdin
* ID_NAME need not be set



git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@729 e04f4502-82db-0310-b1af-f799f365da79
b89a21b
Mike added Java version string to initial output
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@739 e04f4502-82db-0310-b1af-f799f365da79
067b7ce
Mike refactored for modularity
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@765 e04f4502-82db-0310-b1af-f799f365da79
b13206d
Mike initial support for module-based ingestion
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@766 e04f4502-82db-0310-b1af-f799f365da79
c53f6f2
Mike cleanup from yesterday's changes
loading small files from a large filesystem now works more smoothly


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@768 e04f4502-82db-0310-b1af-f799f365da79
888b298
Mike fixed INPUT_PATH
removed "private" from fields that should be subclassable


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@769 e04f4502-82db-0310-b1af-f799f365da79
4d0bdd0
Mike refactoring - no release jar
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@770 e04f4502-82db-0310-b1af-f799f365da79
803035d
Mike one more tweak
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@771 e04f4502-82db-0310-b1af-f799f365da79
0456f9d
Mike more refactoring
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@772 e04f4502-82db-0310-b1af-f799f365da79
9baca92
Mike various tweaks and enhancements
* ID_NAME=#DOCUMENT
* .gz support (but no tar support)



git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@777 e04f4502-82db-0310-b1af-f799f365da79
8d76086
Mike fixed an infinite-loop bug (truncated XML with FATAL_ERRORS=false)
improved memory-efficiency and fd-efficiency of zipfile handling



git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@780 e04f4502-82db-0310-b1af-f799f365da79
4924a06
Mike added delimited-data loader
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@786 e04f4502-82db-0310-b1af-f799f365da79
b68f4f5
Mike added support for LANGUAGE and NAMESPACE
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@797 e04f4502-82db-0310-b1af-f799f365da79
6f1fa39
Mike fix an inconsistency in the README
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@798 e04f4502-82db-0310-b1af-f799f365da79
2bbe3d3
Mike fixed a bug with non-XML format specifications
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@801 e04f4502-82db-0310-b1af-f799f365da79
cc432e5
Mike added docs for USE_FILENAME_COLLECTION
fixed buggy USE_FILENAME_COLLECTION behavior
exposed more fields for subclassing
documented 65-kB limit for XML-STRING technique



git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@805 e04f4502-82db-0310-b1af-f799f365da79
7c813c7
Mike 2008-06-17.2
* fixed binary ingestion truncation, reported by Paul P.



git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@811 e04f4502-82db-0310-b1af-f799f365da79
057ce48
Mike fix for RECORD_NAMESPACE, reported by Dave Sherman
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@814 e04f4502-82db-0310-b1af-f799f365da79
e5a810c
Mike 2008-06-24.1
Fix for filedescriptor leak
Refactoring


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@817 e04f4502-82db-0310-b1af-f799f365da79
613e1b0
Mike fixed input encoding for streaming content
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@847 e04f4502-82db-0310-b1af-f799f365da79
7d4840c
Mike remove call to System.exit
add Solaris-specific troubleshooting


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@858 e04f4502-82db-0310-b1af-f799f365da79
b121fa5
Mike tried to improve docs and error message for ID_NAME and RECORD_NAME
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@871 e04f4502-82db-0310-b1af-f799f365da79
48e5355
Mike refactor for pluggable input handlers
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@885 e04f4502-82db-0310-b1af-f799f365da79
eed4529
Mike remove 65-kB limit (that was easy!)
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@895 e04f4502-82db-0310-b1af-f799f365da79
6be22e2
Mike added byte throttling
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@896 e04f4502-82db-0310-b1af-f799f365da79
2d6279d
Mike fixes buggy interaction of FileLoader and XccModuleContent
improves throttling


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@897 e04f4502-82db-0310-b1af-f799f365da79
f4d20a3
Mike refactoring - better properties handling
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@899 e04f4502-82db-0310-b1af-f799f365da79
3593df6
Mike added IGNORE_FILE_BASENAME
implemented reflection-based config defaults
improved XccModuleContent handling of existence checks


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@900 e04f4502-82db-0310-b1af-f799f365da79
449fb6f
Mike fixed INPUT_STRIP_PREFIX
added extra queue log messages
refactored XCC dependencies


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@902 e04f4502-82db-0310-b1af-f799f365da79
ba0a499
Mike added INPUT_FILE_SIZE_LIMIT
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@903 e04f4502-82db-0310-b1af-f799f365da79
e1546b0
Mike fix from James Clippinger
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@904 e04f4502-82db-0310-b1af-f799f365da79
dd18436
Mike publishing latest build
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@905 e04f4502-82db-0310-b1af-f799f365da79
ebb93a9
Mike better reference counting for zip archives
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@956 e04f4502-82db-0310-b1af-f799f365da79
6c844cc
Mike added two new properties
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@961 e04f4502-82db-0310-b1af-f799f365da79
ac40db7
Mike added missing copyrights
experimental support for LOOP_FOREVER and DELETE_INPUT_FILES


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1245 e04f4502-82db-0310-b1af-f799f365da79
4c79cd9
Mike solaris packages
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1248 e04f4502-82db-0310-b1af-f799f365da79
01e6f6e
Mike bypass throttling when skipping records
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1251 e04f4502-82db-0310-b1af-f799f365da79
75a4dae
Mike fix for non-utf8 environments
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1254 e04f4502-82db-0310-b1af-f799f365da79
4a0a7b3
Mike ssl support
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1259 e04f4502-82db-0310-b1af-f799f365da79
2d4d702
Mike improved forest placement
experimental: pluggable producers


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1271 e04f4502-82db-0310-b1af-f799f365da79
8b35335
Mike base collections documentation
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1274 e04f4502-82db-0310-b1af-f799f365da79
bb74085
Mike improved XML escaping
unescape xpp jarfile path
ignore case for repair option
mutex around zip reference counts


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1301 e04f4502-82db-0310-b1af-f799f365da79
347484a
Mike bug fixes
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1310 e04f4502-82db-0310-b1af-f799f365da79
e506795
Mike refactoring
http module class
another charsetencoder bug fix


git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1311 e04f4502-82db-0310-b1af-f799f365da79
5eccd90
Mike maven support, via Mark Helmstetter
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1348 e04f4502-82db-0310-b1af-f799f365da79
1d5dc37
Mike WIP - new role capabilities
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1354 e04f4502-82db-0310-b1af-f799f365da79
d434e49
Mike publishing new role capabilities
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1355 e04f4502-82db-0310-b1af-f799f365da79
f1f36d3
Mike Fix misplaced warning
git-svn-id: http://developer.marklogic.com/svn/recordloader/trunk@1356 e04f4502-82db-0310-b1af-f799f365da79
6591570
Michael Blakeley new 0664f16
Michael Blakeley new 64499ad
Michael Blakeley to gh-pages 46bf7ba
Michael Blakeley x e1e1bc0
Michael Blakeley x e89fc07
Michael Blakeley encoding improvements 6395700
Michael Blakeley tweaks bc21099
Michael Blakeley throttling fixes b3abcd1
Michael Blakeley trim RECORD_NAME and RECORD_NAMESPACE e00af65
Michael Blakeley DDL improvements db14ef6
Michael Blakeley DDL improvements, round 2 88bc918
Michael Blakeley fix for thread interrupt problems 33cfcbd
Michael Blakeley trim classname properties 92aaca3
Michael Blakeley fix for binary inputs 8aff274
@eedeebee eedeebee Empty commit 18dc2d4
@eedeebee eedeebee Merge branch 'master' of github.com:marklogic/recordloader 3dfe486
@eedeebee eedeebee Empty Commit 64f878e
@mblakele mblakele fixed ant config and startup NPE 7401c38
@mblakele mblakele fix for escaped quotes in attribute values 5880aeb
@mblakele mblakele Merge pull request #3 from mblakele/master
fix for escaped quotes in attribute values
b5c441d
@prestonmcgowan prestonmcgowan Added ENCRYPTED_PASSWORD capability e5004cf
@prestonmcgowan prestonmcgowan github is forcing me a1d4a4c
Contributor

HsiaoSu commented Feb 24, 2012

This looks good. But please add these:

  1. Copyright notices as headers on all new files. Just copy them from exisitng files should be fine.
  2. Some documentation on gh-pages on how to use this. I think the idea is to run one of the utilities to generate the encrypted password, put the encrypted password in the .properties file, set the ENCRYPTED_PASSWORD flag, then go. But I'm not sure which utility to run.

Thanks for contributing this!

Hsiao

@HsiaoSu HsiaoSu merged commit f71a2c0 into marklogic-community:master Mar 13, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment