These DemoDev projects illustrate techniques for building Java based components, and is a collection of Java projects forms the nucleus of an upcoming book on Java programming. Service Oriented Architecture examples are included as well. Topics included are:
- Various object-oriented and functional coding techniques are illustrated with real examples.
- Ready-to-uses utilities as libraries and executables.
- Performance testing approaches.
- DevOps examples.
- Interview questions.
One sub-topic is a collection of interviewing coding questions is included to illustrate industry-wide assessment of Java coding skills. Often these coding solutions do not follow programming best practices in order to meet a time or complexity constraint in an interview. However, the other sub-projects follow well known programming principles. Please see https://medium.com/better-programming/kiss-dry-and-code-principles-every-developer-should-follow-b77d89f51d74 for an overview, and scan the Sub-Project Structure description documented below.
Algorithms and performance testing approaches are included in the repository, along with ready-to-use utility programs and libraries. There is also a DevOps sub-project collection of build related utilities and templating.
The Java projects in DemoDev use Maven to build and JUnit to test artifacts. Note that JUnit is a framework, one of many, for regression testing. A Maven build project is defined by an XML structure persisted in a "pom.xml" file located in the build project directory. Maven build projects may be nested in a tree structure.
In a tree of Maven build projects, the root (top-most) parent build project defines common MAVEN dependencies and version definitions in a pom.xml. Nested child build projects are located under the top-level parent build project, each child with their own pom.xml file, optionally overriding inherited MAVEN dependencies and versions.
A child Maven build project build may, in turn, be a parent of an even lower level child build project, thus forming a multi-level tree of related build projects, each inheriting common MAVEN dependencies.
Maven creates a build using "plugins" that implement build actions. Similar to dependencies and versions, plugins are included in parent build projects and optionally overridden in child build projects.
There are far too many Maven tutorials to mention here, so please search the web for one that helps you. More information is provided in the Getting Started section below.
Most of DemoDev is built using Java 8, but Java 9 introduced the REPL (Run-Execute-Print-Loop); an interactive Java runtime environment. Many of us are still tied to Java 8 for business reasons, but would like to experiment with the REPL. This site, called https://github.com/javaterminal/tryjshell, offers a browser-accessible version of the Java REPL. They also provide a hosted version at https://tryjshell.org/.
Please contact Donald Trummell via Email using dtrummell@gmail.com for additional information regarding commercial use of projects in this repository. His LinkedIn profile is found at http://www.linkedin.com/in/donaldtrummell/. You may review the published form of this documentation at http://donaldet.github.io/DemoDev/.
This project contributes utility and algorithmic components to the Wind Power Explorer project, with permission (please see the https://github.com/WindPowerExplorer/FullSail repository).
All the information needed to build these projects is detailed in the BuildResources document described below. The process you will follow is:
- Download required build dependencies (e.g., Java, MAVEN, and optionally ANT)
- Install and test the build dependencies
- Fork or download the source code from the GitHub repository (see the DemoDev repository at https://github.com/DonaldET/DemoDev for source code access.)
- On the command line, from the downloaded base directory, execute: mvn clean install
The StaticResources directory references instructions on how to setup a MAVEN and JAVA build environment in document BuildResources.pdf. The PDF is a GITHUB hosted document that also contains links back to this source code repository (DemoDev).
DemoDev is composed of multiple sub-projects, with each sub-project having a README.md file offering information about the sub-project and its contents. Interesting sub-projects include:
- algorithms: Math algorithm demonstrations.
- optimizations: Useful utilities around optimizing processes.
- hacks: Unusual solutions to interesting problems that don't necessarily reflect best practices.
- jerseyservices: RESTful Java service implementations using the Jersey framework
- memoryindexing: In-memory search algorithms
- utils: Utilities used by DemoDev and related projects (e.g., code generation used by WindPower Explorer)
- codingexams: Coding and Algorithm interview questions asked over a whiteboard or in a collaboration environment coding environment.
Sub-projects usually include a business motivation for the coding examples in this section of the repository. The examples also show good OO design principals, such as KISS, DRY (don't repeat yourself), and SOLID (Single responsibility, Open-closed, Liskov substitution, Interface segregation and Dependency inversion.) This site violates YAGNI for educational purposes. Spring has a very readable overview of the Liskov Substitution Principle (see https://springframework.guru/principles-of-object-oriented-design/liskov-substitution-principle/.) While not directly illustrated in these examples, an important principle is the principle of least astonishment (or least surprise.) Related to KISS, it really means that a code reader is not surprised by the approach taken to solve a problem (see https://en.wikipedia.org/wiki/Principle_of_least_astonishment.)
The sub-projects have their JavaDoc entries explaining their role in the solution and the portion of the problem they address. Many sub-projects include JUnit tests because this reflects good TDD practice and allows maintenance of the examples over time. Stability over time is accomplished by using the regression aspect of these tests.
Many of the projects appear unnecessarily complicated by use of the Spring framework to solve what may appear to be a potentially simple problem. This is done to illustrate Spring usage as well as the basic techniques shown in the example. Overall, a professional Software Engineer working on production code would provide background for code implementation choices, some guidelines on deployment and usage in the JavaDoc entries, and tips to help maintain the code. The example invocations within unit tests help in achieving these documentation goals.
-
See Learning-Resources for tools to improve Java, Python, and SQL skills.
-
Click Welcome.
These Mathematical Algorithms show methods of performance testing and demonstrate coding principles like:
- Separation of interface and implementation
- Use of multiple implementations (Strategy and Command Patterns)
- JUnit tests where feasible
| Project | Description |
|---|---|
| badaddr | Demonstrates uncontrolled relative errors from floating point addition and the Kahn algorithm for mitigating some of the errors. This is the basis for the article: https://www.linkedin.com/pulse/just-doesnt-add-up-part-1-donald-trummell/. |
| cpuconcurrent | Compare Java 8 streaming performance of forEach and collect approaches to computing using multiple threads. We will add a Python version using multiple processes to concurrently compute the same values. This is the basis of the LinkedIn article TBD. See http://mathforum.org/dr.math/faq/faq.property.glossary.html for background on the mathematical properties of addition. |
| largenumeric | Example of the Kahn Addition Algorithm for mitigating some of the errors associated with a long sequence of additions. Computer floating point addition is neither commutative nor associative. |
| nasa-sensor | Monitor sensor radiation exposure with multiple algorithms; trade off space and time. This is the basis of the article: https://www.linkedin.com/pulse/needs-really-fast-donald-trummell/. |
| quadradic | Example of a Quadradic Equation solver using both direct and iterative methods. |
| sieve | Implements the classical method to find primes in an interval using the Sieve of Eratosthenes.## Sub-Project Big Data |
TBD.
| Project | Description |
|---|---|
| tbd | TBD |
These short examples demonstrate "less than the best" coding practices that are sometimes used to get around a solution constraint until a better solution can be found.
| Project | Description |
|---|---|
| serialhack | Use serialization to assign value to a private field lacking a setter |
These examples demonstrate in-memory searching algorithms that stress simple approaches where possible.
| Project | Description |
|---|---|
| simple linear search | Use specially formatted names and attributes and perform "brute force" linear search . . . it is both simple and fast |
| top X query | Compares the performance of priority queue and sorted array based scanners on "top X" queries across a random collection of Game instances. Game instances have an ascending comparator that allows the instances to be ordered on several attributes. The "Top X" query returns a sorted collection of the desired subset of Game instances. |
The top X query example uses the motivation of finding the (say) top 10 scores of a complex game where several game attributes allow for unique ordering. We vary the number of inputs to scan and the size of the top set. Since 100 elements is a large list, that is our upper limit for this performance test.
These examples demonstrate NoSQL persistence.
| Project | Description |
|---|---|
| TBD | TBD |
DevOps projects are not directly used by DemoDev and related projects, but illustrate good development environment practices and useful Templates for creating those environments.
| Project | Description |
|---|---|
| Dependencies | A sample Python project (myapppy), with PyBuilder and PyCharm, is set up to mimic Java development environments (Maven and Eclipse). The project enables creation of binary deployable artifacts. This utility is the basis of the DevOps article: https://www.linkedin.com/pulse/setting-up-python-project-virtual-environment-pycharm-donald-trummell. |
The DevOps projects are templates meant to be forked (copied) and modified.
These projects are working code used by DemoDev and related projects. They illustrate good engineering practices and represent useful utilities.
| Project | Description |
|---|---|
| category-optimizer | This is both a Java and Python based library and utility program that finds score boundaries in a collections of scores that optimizes the R-Squared value of the resulting categories. |
| DemoGenerator | A Freemarker based text generation utility supporting configurable symbol interpolation, composible context/model properties specifications, composable template specifications, and driven by command-line parameters. This utility is used by the WindPowerExplorer repository to create Oozie properties files and HQL table specifications. It is also the basis of the DevOps article: https://www.linkedin.com/pulse/apache-ant-devops-practices-donald-trummell. |
Note: These utilities are distributed using Java JAR files and invoked as executable Jars. The Python versions are created by PyBuilder and typically distributed as wheel files.
These coding exams are elaborations on topics discussed during interviews and "take home" coding problems. Some problems were from educational sites offering coding practice. The resulting solutions demonstrate a "professional engineering" approach to simple coding questions asked in these situations. Solutions presented here include MAVEN builds, JUnit tests, and JavaDoc. Many of these problems were originally done under time constraints and considerable cleanup was required before presentation here in DemoDev.
These solutions also demonstrate coding principles like:
- Separation of interface and implementation.
- Use of multiple implementations (e.g., Strategy and Command Patterns.)
- Test driven development (shown by reviewing testing order; simple initial conditions to complex corner cases.)
- JUnit framework tests where feasible, or custom test cases when created in an interviewing environment.
| Client | Project | Description |
|---|---|---|
| Amazon | codingchallange | Create a recommendation feature (recommendation engine) called "Games Your Friends Play". The recommendation logic is based on the following rules: a customer should only be recommended games that their friends own but they don"t; the recommendations priority is driven by how many friends own a game - if multiple friends own a particular game, it should be higher in the recommendations than a game that only one friend owns. |
| Amazon | findmedian | Illustrate two methods of finding the median of a (non-infinite) stream of integers. One method is highly optimized compared to the other. |
| Amazon | infrastructure | We solve two problems: find the minimum times to complete a list of tasks in dependency order; find the longest substring of a string, with the substring having k or fewer unique characters. |
| Amazon | invoiceparser | Scan an Amazon EBook invoice and extract date, title, type and amount of book purchase. |
| Amazon | rangeconsolidator | Scan a list of integer ranges and merge overlapping ranges, collapsing them into an all-inclusive range. Performance test as well. This is the basis of the article: https://www.linkedin.com/pulse/lies-damn-algorithm-analysis-donald-trummell/. |
| Amazon | statemachines | Find filled bins in a packing structure; dynamically monitor the number of active servers in a cluster. |
| AppDynamics | top X query | Query the top 10 game scores using a priority queue; note that this is in sub-project MemoryIndexing. |
| Apple | compounditerator | Concatenate data sources using a list of iterators for each data source. |
| Apple | factorial | Contrast two factorial algorithms. |
| Cisco | sieve | Capture the top X elements of a random integer array using four techniques. |
| CreditKarma | list dependencies | Given a list of modules names, each with a list of named dependencies, output a list with dependencies of a module appearing before the module itself. |
| Cruise-Control | task processor | Process tasks sequentially with parameterizable cooling time and fixed execution times. |
| EasyPost | pick | pick create-pick event pairs from an input stream and summarize times by location. |
| Elance | diagonal | Square matrix diagonal elements list generator. |
| Gap | gap cart | A sample shopping cart implementation with pricing strategies. |
| GE | gesail twowriters | Apply two threads alternately incrementing a counter; uses multiple counter implementations, and includes unit tests and a runnable jar. |
| GE | geturner binarysearch | Compare a recursive and iterative solution to the classical binary search algorithm; includes example of the template pattern. |
| Gesicht Buch | Infrastructure | Minimum Task Timer, Longest Substring with Unique Characters, Stream to Lines, and Sum of three nearest target. |
| Grand Tour | mars rover | Simulate a planetary rover vehicle remote command facility. |
| Granular | Sudoku Game Checker | Check rows, columns, and 3x3 cells for duplicates. |
| IJSDE | Problems | Programming skills validation examples associated with a job-placement web-site. |
| Liveramp | autoboxing | Explores consequences of Java "autoboxing" of primative values, and examines integer comparison in the context of a Java Comparator. This is the basis of the article: https://www.linkedin.com/pulse/pilot-error-java-autoboxing-donald-trummell/. |
| Liveramp | bitsearch | We must count unique 32 bit client IPs accessing our web-site over some time period (a day let’s say). This is the basis of my article: https://www.linkedin.com/pulse/test-driven-development-tdd-really-works-donald-trummell-1c/. The article show cases TDD., and will lead to an article about generating test data (e.g.., see https://queue.acm.org/detail.cfm?id=3355565.) |
| Lumendata | duplicatechecker | Finds duplicate column values in a row. |
| PINTR | expand | Write a function that expands a HashMap to an array. The HashMap contains a key and a value, from which your function will produce an array with the key repeated the respective number of times. Output order of the array does not matter. |
| Rakuten | Merge | Merge two ordered arrays of integers, in place, using space O(1). |
| Socotra | ordered | Print by first by frequency, but then by first appearance for tie. |
| Trulia | treesearch | Find the "second largest value" in a specially constructed binary tree. |
| WeWork | spiral | Spiral-print a matrix (from LeetCode). |
| WeWork | patterns | Examples of Strategy, Factory, Bridge, and Decorator patterns. |
These web-sites are great places to learn the art of programming by actually doing programming exercises. We also recommend some helpful books.
Here are some educational web sites for testing your Java and Python coding skills:
- https://www.hackerrank.com/, test problems and competitive scoring.
- https://leetcode.com/, presenting problems and solutions in a peer-reviewed development environment.
- https://app.codility.com/programmers/, presenting training examples and a multi-language testing environment; primarily used as a developer skills testing environment.
- https://coderpad.io/, a white-board development environment for coding exams.
- https://app.codility.com/programmers/, like coderpad, the codility web site is a testing environment offering practice problems.
- https://www.pramp.com/#/, an interactive coding interview environment for practicing coding interviews.
Support for testing SQL programming, which is heavily used in Data Engineering, includes these sites that run MySQL sandboxes:
- Online SQL teaching instances: http://sqlfiddle.com/ and https://www.freesqldatabase.com/.
- SQL runner only: https://paiza.io/projects/featured?language=mysql.
- SQL skills testing: https://coderpad.io/demo and the practice sandbox https://coderpad.io/sandbox.
Additional SQL learning resources include:
- The W3Schools editor https://www.w3schools.com/sql/trysql.asp?filename=trysql_op_in.
- A free online course is found at: https://www.codecademy.com/learn/learn-sql.
- An Oracle teaching link: https://livesql.oracle.com/apex/f?p=590:1000 (see https://www.thatjeffsmith.com/archive/2016/03/a-place-to-learn-oracle-no-setup-required/.
- A link discussing free courses: https://javarevisited.blogspot.com/2015/06/5-websites-to-learn-sql-online-for-free.html.
Finally, much of the Python content of this repository is concerned with Apache Spark, and a good free Spark implementation is hosted by Databricks (the community edition) found at https://databricks.com/try-databricks.## Build Information Naming and structural conventions, along with collected statistics on Maven build structure. Key Maven plugins are documented as well.
Projects have a package structure that reflects potential use outside of the DemoDev repository ("don") and for repository use only ("demo").
- Packaging Conventions - "don.*"
D:\GitHub\DemoDev>findstr /srb "package " *.java | find "package don" > pkg_don.log
D:\GitHub\DemoDev>find /C "package" pkg_don.log
---------- PKG_DON.LOG: 111
- Packaging Conventions - "demo.*"
D:\GitHub\DemoDev>findstr /srb "package " *.java | find "package demo" > pkg_demo.log
D:\GitHub\DemoDev>find /C "package" pkg_demo.log
---------- PKG_DEMO.LOG: 231
- Packaging Conventions - Other:
D:\GitHub\DemoDev>findstr /srb "package " *.java | find /V "package don" | find /V "package demo" > pkg_other.log
D:\GitHub\DemoDev>find /C "package" pkg_other.log
---------- PKG_OTHER.LOG: 0
The plugins are used in the build structure to achieve build tasks.
| Plugin | Description | Link |
|---|---|---|
maven-compiler-plugin |
Defines Javac compiler source and runtime Java versions | https://maven.apache.org/plugins/maven-compiler-plugin/ |
maven-resources-plugin |
Copies source and test resources to output | https://maven.apache.org/plugins/maven-resources-plugin/ |
maven-jar-plugin |
Packages code and resources into a Jar with a Manifest | https://maven.apache.org/plugins/maven-jar-plugin/ |
maven-surefire-plugin |
Runs the application tests based on testing dependencies | https://maven.apache.org/surefire/maven-surefire-plugin/ |
TBD