I've traced down a race condition after a few hours and found the fix. So I wonder if this can be included in the default configuration of the ConnectionProperties for H2EmbeddedDatabaseConfigurer.
Per default DB_CLOSE_ON_EXIT is true. Thus, if the test VM exits, the Spring ApplicationContext gets closed, but H2 also registers a shutdown hook: the DatabaseCloser thread shuts down the database in parallel. As a consequence, all scripts configured via <jdbc:script location="..." execution="DESTROY" /> (that should be run at closing time) fail because the database was already shutdown.
It would be good if the following line in H2EmbeddedDatabaseConfigurer.configureConnectionProperties() ...
This commit sets the DB_CLOSE_ON_EXIT flag to false for embedded H2
databases loaded using H2EmbeddedDatabaseConfigurer (i.e., via Spring's
<jdbc:embedded-database /> XML namespace, EmbeddedDatabaseBuilder,
EmbeddedDatabaseFactory, and EmbeddedDatabaseFactoryBean).