Skip to content

Commit ad0229b

Browse files
bajtosbajtos
committed
feat(repository): migrateSchema APIs
Introduce a new Application-level method `app.migrateSchema()` provided by `RepositoryMixin`, this method executes schema migration as implemented by datasources registered with the application. Simplify the instructions shown in `Database-migrations.db` Add an example `migrate.ts` script to our Todo example to verify that the code snippet shown in the docs works as intended.
1 parent 7be76a4 commit ad0229b

File tree

5 files changed

+299
-99
lines changed

5 files changed

+299
-99
lines changed

‎docs/site/Database-migrations.md‎

Lines changed: 78 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -20,123 +20,111 @@ LoopBack offers two ways to do this:
2020
- **Auto-update**: Change database schema objects if there is a difference
2121
between the objects and model definitions. Existing data will be kept.
2222

23-
## Implementation Example
23+
{% include warning.html content="Auto-update will attempt to preserve data while
24+
updating the schema in your target database, but this is not guaranteed to be
25+
safe.
2426

25-
Below is an example of how to implement
26-
[automigrate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-automigrate)
27-
and
28-
[autoupdate()](http://apidocs.loopback.io/loopback-datasource-juggler/#datasource-prototype-autoupdate),
29-
shown with the
30-
[TodoList](https://loopback.io/doc/en/lb4/todo-list-tutorial.html) example.
27+
Please check the documentation for your specific connector(s) for a detailed
28+
breakdown of behaviors for automigrate! " %}
3129

32-
Create a new file **src/migrate.ts** and add the following import statement:
30+
## Examples
3331

34-
```ts
35-
import {DataSource, Repository} from '@loopback/repository';
36-
```
37-
38-
Import your application and your repositories:
32+
LoopBack applications are typically using `RepositoryMixin` to enhance the core
33+
`Application` class with additional repository-related APIs. One of such methods
34+
is `migrateSchema`, which iterates over all registered repositories and asks
35+
them to migrate their schema. Repositories that do not support schema migrations
36+
are silently skipped.
3937

40-
```ts
41-
import {TodoListApplication} from './index';
42-
import {TodoRepository, TodoListRepository} from './repositories';
43-
```
38+
In the future, we would like to provide finer-grained control of database schema
39+
updates, learn more in the GitHub issue
40+
[#487 Database Migration Management Framework](https://github.com/strongloop/loopback-next/issues/487)
4441

45-
Create a function called _dsMigrate()_:
42+
### Auto-update database at start
4643

47-
```ts
48-
export async function dsMigrate(app: TodoListApplication) {}
49-
```
50-
51-
In the _dsMigrate()_ function, get your datasource and instantiate your
52-
repositories by retrieving them, so that the models are attached to the
53-
corresponding datasource:
54-
55-
```ts
56-
const ds = await app.get<DataSource>('datasources.db');
57-
const todoRepo = await app.getRepository(TodoRepository);
58-
const todoListRepo = await app.getRepository(TodoListRepository);
59-
```
44+
To automatically update the database schema whenever the application is started,
45+
modify your main script to execute `app.migrateSchema()` after the application
46+
was bootstrapped (all repositories were registered) but before it is actually
47+
started.
6048

61-
Then, in the same function, call _automigrate()_:
49+
{% include code-caption.html content="src/index.ts" %}
6250

6351
```ts
64-
await ds.automigrate();
65-
```
66-
67-
This call to automigrate will migrate all the models attached to the datasource
68-
db. However if you want to only migrate some of your models, add the names of
69-
the classes in the first parameter:
52+
export async function main(options: ApplicationConfig = {}) {
53+
const app = new TodoListApplication(options);
54+
await app.boot();
55+
await app.migrateSchema();
56+
await app.start();
7057

71-
```ts
72-
// Migrate a single model
73-
ds.automigrate('Todo');
74-
```
58+
const url = app.restServer.url;
59+
console.log(`Server is running at ${url}`);
7560

76-
```ts
77-
// Migrate multiple models
78-
ds.automigrate(['Todo', 'TodoList']);
61+
return app;
62+
}
7963
```
8064

81-
The implementation for _autoupdate()_ is similar. Create a new function
82-
_dsUpdate()_:
83-
84-
```ts
85-
export async function dsUpdate(app: TodoListApplication) {
86-
const ds = await app.get<DataSource>('datasources.db');
87-
const todoRepo = await app.getRepository(TodoRepository);
88-
const todoListRepo = await app.getRepository(TodoListRepository);
65+
### Auto-update the database explicitly
8966

90-
await ds.autoupdate();
91-
}
92-
```
67+
It's usually better to have more control about the database migration and
68+
trigger the updates explicitly. To do so, you can implement a custom script as
69+
shown below.
9370

94-
The completed **src/migrate.ts** should look similar to this:
71+
{% include code-caption.html content="src/migrate.ts" %}
9572

9673
```ts
97-
import {DataSource, Repository} from '@loopback/repository';
98-
import {TodoListApplication} from './index';
99-
import {TodoRepository, TodoListRepository} from './repositories';
74+
import {TodoListApplication} from './application';
10075

101-
export async function dsMigrate(app: TodoListApplication) {
102-
const ds = await app.get<DataSource>('datasources.db');
103-
const todoRepo = await app.getRepository(TodoRepository);
104-
const todoListRepo = await app.getRepository(TodoListRepository);
76+
export async function migrate(args: string[]) {
77+
const dropExistingTables = args.includes('--rebuild');
78+
console.log('Migrating schemas (%s)', rebuild ? 'rebuild' : 'update');
10579

106-
await ds.automigrate();
80+
const app = new TodoListApplication();
81+
await app.boot();
82+
await app.migrateSchema({dropExistingTables});
10783
}
10884

109-
export async function dsUpdate(app: TodoListApplication) {
110-
const ds = await app.get<DataSource>('datasources.db');
111-
const todoRepo = await app.getRepository(TodoRepository);
112-
const todoListRepo = await app.getRepository(TodoListRepository);
113-
114-
await ds.autoupdate();
115-
}
85+
migrate(process.argv).catch(err => {
86+
console.error('Cannot migrate database schema', err);
87+
process.exit(1);
88+
});
11689
```
11790

118-
Finally, in **src/index.ts**, import and call the _dsMigrate()_ or _dsUpdate()_
119-
function:
91+
After you have compiled your application via `npm run build`, you can update
92+
your database by running `node dist/src/migrate` and rebuild it from scratch by
93+
running `node dist/src/migrate --rebuild`. It is also possible to save this
94+
commands as `npm` scripts in your `package.json` file.
12095

121-
```ts
122-
import {TodoListApplication} from './application';
123-
import {ApplicationConfig} from '@loopback/core';
124-
125-
// Import the functions from src/migrate.ts
126-
import {dsMigrate, dsUpdate} from './migrate';
96+
### Implement additional migration steps
12797

128-
export {TodoListApplication};
98+
In some scenarios, the application may need to define additional schema
99+
constraints or seed the database with predefined model instances. This can be
100+
achieved by overriding the `migrateSchema` method provided by the mixin.
129101

130-
export async function main(options: ApplicationConfig = {}) {
131-
const app = new TodoListApplication(options);
132-
await app.boot();
133-
await app.start();
102+
The example below shows how to do so in our Todo example application.
134103

135-
const url = app.restServer.url;
136-
console.log(`Server is running at ${url}`);
104+
{% include code-caption.html content="src/application.ts" %}
137105

138-
// The call to dsMigrate(), or replace with dsUpdate()
139-
await dsMigrate(app);
140-
return app;
106+
```ts
107+
import {TodoRepository} from './repositories';
108+
// skipped: other imports
109+
110+
export class TodoListApplication extends BootMixin(
111+
ServiceMixin(RepositoryMixin(RestApplication)),
112+
) {
113+
// skipped: the constructor, etc.
114+
115+
async migrateSchema(options?: SchemaMigrationOptions) {
116+
// 1. Run migration scripts provided by connectors
117+
await super.migrateSchema(options);
118+
119+
// 2. Make further changes. When creating predefined model instances,
120+
// handle the case when these instances already exist.
121+
const todoRepo = await this.getRepository(TodoRepository);
122+
const found = await todoRepo.findOne({where: {title: 'welcome'}});
123+
if (found) {
124+
todoRepo.updateById(found.id, {isComplete: false});
125+
} else {
126+
await todoRepo.create({title: 'welcome', isComplete: false});
127+
}
128+
}
141129
}
142130
```

‎examples/todo/src/migrate.ts‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
// Copyright IBM Corp. 2017,2018. All Rights Reserved.
2+
// Node module: @loopback/example-todo
3+
// This file is licensed under the MIT License.
4+
// License text available at https://opensource.org/licenses/MIT
5+
6+
import {TodoListApplication} from './application';
7+
8+
export async function migrate(args: string[]) {
9+
const existingSchema = args.includes('--rebuild') ? 'drop' : 'alter';
10+
console.log('Migrating schemas (%s existing schema)', existingSchema);
11+
12+
const app = new TodoListApplication();
13+
await app.boot();
14+
await app.migrateSchema({existingSchema});
15+
16+
// Connectors usually keep a pool of opened connections,
17+
// this keeps the process running even after all work is done.
18+
// We need to exit explicitly.
19+
process.exit(0);
20+
}
21+
22+
migrate(process.argv).catch(err => {
23+
console.error('Cannot migrate database schema', err);
24+
process.exit(1);
25+
});

‎packages/repository/src/datasource.ts‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
// This file is licensed under the MIT License.
44
// License text available at https://opensource.org/licenses/MIT
55

6-
import {AnyObject} from './common-types';
6+
import {AnyObject, Options} from './common-types';
77
import {Connector} from './connectors';
88

99
/**
@@ -17,3 +17,21 @@ export interface DataSource {
1717
// tslint:disable-next-line:no-any
1818
[property: string]: any; // Other properties that vary by connectors
1919
}
20+
21+
export interface SchemaMigrationOptions extends Options {
22+
/**
23+
* When set to 'drop', schema migration will drop existing tables and recreate
24+
* them from scratch, removing any existing data along the way.
25+
*
26+
* When set to 'alter', schema migration will try to preserve current schema
27+
* and data, and perform a non-destructive incremental update.
28+
*/
29+
existingSchema?: 'drop' | 'alter';
30+
31+
/**
32+
* List of model names to migrate.
33+
*
34+
* By default, all models are migrated.
35+
*/
36+
models?: string[];
37+
}

‎packages/repository/src/mixins/repository.mixin.ts‎

Lines changed: 63 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,14 @@
33
// This file is licensed under the MIT License.
44
// License text available at https://opensource.org/licenses/MIT
55

6-
import {Class} from '../common-types';
7-
import {Repository} from '../repositories/repository';
8-
import {juggler} from '../repositories/legacy-juggler-bridge';
6+
import {BindingScope, Binding} from '@loopback/context';
97
import {Application} from '@loopback/core';
10-
import {BindingScope} from '@loopback/context';
8+
import * as debugFactory from 'debug';
9+
import {Class} from '../common-types';
10+
import {juggler, Repository} from '../repositories';
11+
import {SchemaMigrationOptions} from '../datasource';
12+
13+
const debug = debugFactory('loopback:repository:mixin');
1114

1215
/**
1316
* A mixin class for Application that creates a .repository()
@@ -163,6 +166,46 @@ export function RepositoryMixin<T extends Class<any>>(superClass: T) {
163166
}
164167
}
165168
}
169+
170+
/**
171+
* Update or recreate the database schema for all repositories.
172+
*
173+
* **WARNING**: By default, `migrateSchema()` will attempt to preserve data
174+
* while updating the schema in your target database, but this is not
175+
* guaranteed to be safe.
176+
*
177+
* Please check the documentation for your specific connector(s) for
178+
* a detailed breakdown of behaviors for automigrate!
179+
*
180+
* @param options Migration options, e.g. whether to update tables
181+
* preserving data or rebuild everything from scratch.
182+
*/
183+
async migrateSchema(options: SchemaMigrationOptions = {}): Promise<void> {
184+
const operation =
185+
options.existingSchema === 'drop' ? 'automigrate' : 'autoupdate';
186+
187+
// Instantiate all repositories to ensure models are registered & attached
188+
// to their datasources
189+
const repoBindings: Readonly<Binding<unknown>>[] = this.findByTag(
190+
'repository',
191+
);
192+
await Promise.all(repoBindings.map(b => this.get(b.key)));
193+
194+
// Look up all datasources and update/migrate schemas one by one
195+
const dsBindings: Readonly<Binding<object>>[] = this.findByTag(
196+
'datasource',
197+
);
198+
for (const b of dsBindings) {
199+
const ds = await this.get(b.key);
200+
201+
if (operation in ds && typeof ds[operation] === 'function') {
202+
debug('Migrating dataSource %s', b.key);
203+
await ds[operation](options.models);
204+
} else {
205+
debug('Skipping migration of dataSource %s', b.key);
206+
}
207+
}
208+
}
166209
};
167210
}
168211

@@ -180,6 +223,7 @@ export interface ApplicationWithRepositories extends Application {
180223
): void;
181224
component(component: Class<{}>): void;
182225
mountComponentRepositories(component: Class<{}>): void;
226+
migrateSchema(options?: SchemaMigrationOptions): Promise<void>;
183227
}
184228

185229
/**
@@ -293,4 +337,19 @@ export class RepositoryMixinDoc {
293337
* @param component The component to mount repositories of
294338
*/
295339
mountComponentRepository(component: Class<{}>) {}
340+
341+
/**
342+
* Update or recreate the database schema for all repositories.
343+
*
344+
* **WARNING**: By default, `migrateSchema()` will attempt to preserve data
345+
* while updating the schema in your target database, but this is not
346+
* guaranteed to be safe.
347+
*
348+
* Please check the documentation for your specific connector(s) for
349+
* a detailed breakdown of behaviors for automigrate!
350+
*
351+
* @param options Migration options, e.g. whether to update tables
352+
* preserving data or rebuild everything from scratch.
353+
*/
354+
async migrateSchema(options?: SchemaMigrationOptions): Promise<void> {}
296355
}

0 commit comments

Comments
 (0)