…mentation) to take dictionary with extra parameters allowed as well as methods within this class not to reference config but the variable that was set. Adjusted effected tests.

…WebPublisher_unit as unit test until create folder structure for integration vs unit tests.

…o take dictionary with extra parameters allowed as well as methods within this class not to reference config but the variable that was set. Adjusted effected tests.

… to take dictionary with extra parameters allowed as well as methods within this class not to reference config but the variable that was set. Adjusted effected tests. Removed get_logger method as it wasn't used and we used a different logger then. Added checks for if security wasn't enabled.

… using a topic variable instead. Removed changing name of the consumer_conf to meta_consumer_conf. Added additional logging.

… clearer how similar code is to KafkaPublisher class.

…ould notice this usage).

…) to take dictionary with extra parameters allowed as well as methods within this class not to reference config but the variable that was set. Removed logging import and changed to ClientLogger.

…es sense. Added test for extra arguments via constructor. Fixed test for testing less parameters passed in, because more could have made it through.

…rn values a little better where could, and changed a test of assertTrue to assertEqual so got to see what the actual value was when failed.

…olidate generating the key from UUID, and added a little logging. Consolidated topic generation code too.

… file_id_prefix as it is in half the other places including S3MessageAdapter constructor.

…tra parameters allowed. Refactored out of SqsConsumer the connecting part and put into S3Utils, this left only log_level as class var. Put creating a Queue object into receive_messages (can refactor out if ever need again, but single line didn't seem to warrent its own method). Added debug logging.

…arameters for receive_message so a user could create their own queue or use our connect to do so and pass either in.

… with extra parameters allowed so adjusted the tests and added more with more verification.

…ust looks.

…usted SqsConsumer callback parameters to pass along log_level.

…d example test.

…us assuming if they say "glacier" they mean a client of service type "glacier". Little clearer to the user and in the code. This allows boto to catch the error of wrong service name specified and it gives a nice list of choices. Added else statement too for cases user specifies a type we don't expect, will add tests to.

… a moto version issue. getmoto/moto#3297

…of an error to warning.

…tegration. Change circle ci config to run all onestop-python-client tests.

…ved some blank lines from S3Utils.

…ld one. Trying to figure out how to force it to a newer one.

…nflict between botocore 1.18.11 and moto 2.0.5

… using my local aws config when region was not specified.

… to install boto for pip3 every time.

…iables if credentials yml doesn't exist. Commented out integration task in circleCI config. Since cannot reach registry on cedardevs.

…credentials template.

…le _topic_produce to _topic_publish, as it is in the constructor for KafkaPublisher and KafkaConsumer.

…exception. Otherwise wasn't evaluating the variable within message.

…asn't printing value of variable. Added test for that negative case.

…multiplied instead of divided to get seconds.

…realize was used, looked obsolete.

…ython-client based on branch name. Removed unnecessary comments.

…blishing images.

…g to and from examples hoping unnecessary.

…client code to working directory then install requirements.txt.

…dler and onestop-sme onestop-python-client

… in the jobs section and added the sme and s3 builds to jobs.

… circleci) and changed circleci config.

…changed circleci path for onestop-python-client build to indicate use project root as path. Changed paths in root dockerfile back to ./ instead of / so clearer.

… copied over.

…ent, path is one up which is out of context for this dockerfile (hint root dockerfile does that step). Added comment about in root dockerfile.

…from what can tell it only pulls if the image name changes, such as if you have a version number. We don't do this ATM.

…ns w comment and set security to false (cedar-devs is false), adjusted s3_bucket2 to a real bucket, fixed bug of granule type needing to be GRANULE, and added kafka information.

…d if it metadata_type wasn't GRANULE or COLLECTION.

…d if it metadata_type wasn't GRANULE or COLLECTION. Changed try/raise in consume method to try/finally with same closing of the consumer as before. This way exceptions do fail script, because before was unclear why it wasn't working.

…Name was a Delete type.

…ention in last commit that the AvroDeserializer constructor needed more specific parameter names (think it got updated and our order of params was wrong).

…dict instead of config location. Removed mocking for launch_delete_handler script since have unit tests that cover this now.

…er of params seem to have changed.

…er of params seem to have changed. (forgot intellij's multi select for commits doesn't work)

…so as to get better error reporting.

…on so as to get better error reporting.

…to what they are on cedar-devs.

…was changed a while ago in the code).

…pdated test to reflect/catch.

…ilds with old code and hoping is because it was being built before the onestop-python-client-build.

…require onestop-python-client-test to run and the two script builds to require onestop-python-client-build to run.

…equires field.

… build requires configuration.

…d do a json dumps. Was complaining ParsedRecord doesn't have method serialize. Fixed tests, also added size and versionid to test util message since S3MessageAdapter.transform required.

… to os*, is url on cedardevs.

…Adjusted some log statements.

…ode shouldn't be able to throw any exceptions, the ones that could happen you want to bubble up (like connection problems). Moved the closing of connection after while polling section, since was closing the connection.

… there's a problem with the callback it gets thrown - also the sqs_message.delete is outside try so if a connection problem that ends everything.

…evel.

…not serialize the json payload but instead do a json dumps.

…dict instead of config location.

…tement since no harm in extra params.

…_metadata_type, including changing type var used in S3MessageAdapter.

… methods. Mostly added, removed a redundant one.Adjusted one in deliver reports from error to info.

…nformation, and tried to organize by the automated process vs manual. Second pass to come in next story.

… that package was merged into the avro one and will be removed in the future.

…This is because it was too nested and to change to an enum didn't seem to work with Optional.

…e. Removed it since switched to simply a string.

…r available. Never worked for our code, would need work.