-
-
Notifications
You must be signed in to change notification settings - Fork 535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MikroORM v5 #1623
Comments
@B4nan Awesome! One point, maybe kind of related to schema diffing, could be improving naming strategy capabilities. If I'm not mistaken, there is currently no way of defining naming strategies for PKs, indexes, FKs, and others. (TypeORM for instance has a bit more powerful naming strategies.) Another (maybe a little bit wild) idea is supporting Prisma schema – not sure if doable or reasonable in any way, but it is an interesting way of defining DB schema – it may be a feature that brings some traction, and it may allow using Prisma migrations for people who want it, etc. One more thing on my mind – with the current CLI (and migrations) it is hard to use dynamic configuration provided by some kind of configuration services (for instance fetching a config service from DI and passing that as a config to migrations). You either end up duplicating the config data in And last idea (I hope, haha) is adding the possibility of locking migrations table exclusively when running migrations. It will prevent devs from ever starting multiple migration runs in parallel – this is a common mistake in Docker environments when devs use either k8s init containers or run the migration command on container start. I fix similar to the following one prevents the error (and forces all other containers to wait for their turn): await this.orm.em.transactional(async (em) => {
await em.execute('LOCK TABLE migrations IN EXCLUSIVE MODE');
await this.orm
.getMigrator<Migrator>()
.up({ transaction: em.getTransactionContext() });
}); (This is for Postgres and of course this may not be doable for all drivers.) Well, those are just a few ideas, let me know what you think :) |
Sure, good idea. Currently it lives in the schema helper, so it is possible to override with a custom driver only.
Sounds a bit out of scope, on the other hand entity definition is already abstracted so it could be just about having another metadata provider for this.
It is already possible to export a promise from the ORM config.
Agreed, if this single query is enough for postgres, we can definitely do it at least for postgres. Would be great if users of other drivers could come up with similar solutions for those, I'd expect at least mysql to be capable of this too. |
Hm, I thought I tried that, maybe not. OK, will give it a shot!
Yep, this works for me, exclusive lock indeed makes any other queries to the table wait. MySQL doesn't support transactional DDL but I guess |
The config promise support was added in 4.5 so its very recent addition: #1495 Not sure if we need to support also returning async callback, as this just awaits the returned config... |
I see, perfect! No need for a callback, I think, this should cover pretty much any case and it's even simpler. |
This is my wish list for micro orm v5:
|
One of the things that I would like to add is a named migration file similar to typeorm. Currently afaik , the file names are generated implicitly |
Since MikroORM now supports batched updates, inserts, and deletes it would be great if it would support batched selects (#266) as well, even if it was just for simple |
This would be incredible. Right now I have 20 DataLoaders, being able to eliminate these would be awesome! |
As for me it would be great to have Oracle DB support. |
Implemented as
I said this many times. I never really used typeorm, so can't provide much info about this. Better ask someone else who does. Also as the internals are very different (typeorm is kind of a QB when compared to MikroORM, as there is no UoW/in-memory state management), automatic migrations would never be really performant. Best what I can offer is this commit where I migrated the typeorm version to MikroORM, but it is a bit dated.
Blocked by knex not supporting it. Probably possible to get around, but I am tired of hacking around how knex works. Maybe I will give it a try and remove it as a whole from schema diffing, as the code gets bloated because of this a lot, and I already had to implement better part of things myself anyway. But I'd still like to leverage knex for the new drivers to come before I remove it. You could probably use
The filenames are generated based on configurable callback, and if you adjust the
I was looking into this finally, and after some time playing with this, I finally see how it works. I must say that it feels funny now, all those "N+1 safety" claims, as the dataloader approach is valid only in very specific constraints (e.g. calling Anyway, as mentioned, I do understand how this would be helpful for GQL users, so will try to make it work at least for the reference/collection loading. But not really a priority for me. |
recipe for named migration file will be great |
It's actually really useful for batch processing of entities as well, where you fetch a batch of entities then based on values in each entity fetch different kinds of entities |
Can you provide some code, or a test case ideally? Maybe I just wasn't able to make it work the way it should. All the examples I came up with felt much better written with populate hints so I haven't seen any real benefits when used manually (again, for GQL it's a different story). Only way I was able to make the dataloader to batch things was via |
Would be great to get some feedback on the new strict I also wanted to make the string populate work with dot notation (e.g. If you see more places we could make more strict, I am all ears. Another thing I'd like to hear some feedback on is removal of (not only) boolean parameters in EM and driver API in favour of options objects. Currently most of the methods either have very long signature or support two (or more) signatures, so I'd like to keep only the options parameter. This means that following (and many more) would be gone: -async find(entityName: EntityName<T>, where: FilterQuery<T>, populate?: P, orderBy?: QueryOrderMap, limit?: number, offset?: number): Promise<Loaded<T, P>[]>;
+async find(entityName: EntityName<T>, where: FilterQuery<T>, options?: FindOptions<T, P>): Promise<Loaded<T, P>[]>;
-async findOne(entityName: EntityName<T>, where: FilterQuery<T>, populate?: P, orderBy?: QueryOrderMap): Promise<Loaded<T, P> | null>;
+async findOne(entityName: EntityName<T>, where: FilterQuery<T>, options?: FindOneOptions<T, P>): Promise<Loaded<T, P> | null>;
-nativeInsertMany(entityName: string, data: EntityDictionary<T>[], ctx?: Transaction, processCollections?: boolean, convertCustomTypes?: boolean): Promise<QueryResult>;
+nativeInsertMany(entityName: string, data: EntityDictionary<T>[], options?: InsertManyOptions<T>): Promise<QueryResult>; It would improve readability, allow adding new options, and might as well improve TS performance and error handling (I guess we all know those |
Seeing the improvements for migrations I'd have another one – the possibility to use pure |
I think view entities is an important feature. |
viewEntity exists in typeorm. Is it the same thing that you want? https://typeorm.io/#/view-entities |
It does indeed require a This is probably the most generic example I could give but hopefully it shows what I mean async function main() {
for await (const batchOfData of someStreamOfData) {
await Promise.all(
batchOfData.map(async (data) => {
switch (data.type) {
case 'fooAction':
{
const entity = await em.findOne(Foo, data.id);
entity.someField = data.value;
}
break;
case 'heartbeat':
{
const status = await em.findOne(Status, data.id);
status.lastActive = new Date();
status.logs.add(new LogEntry());
}
break;
}
})
);
await em.flush();
em.clear();
}
} |
Yes, I'm migratting from type orm to mikro orm, and I realize mikro does not suoport views. |
@B4nan I'd like to add some additional thoughts regarding migrations. My team has been working with MikroORM for quite some time now and we have consistently run into some "gotchas" with migrations that mostly stem from using the current database schema as the basis for comparison rather than the current migrations in the code base. Many of us came from Django which uses the existing migrations as the basis for comparison and that results in a much more predictable result when dealing with migrations. The biggest problem on our team is when we are working on multiple branches that have different entities. Many times I'll switch to a branch to assist a team mate and make some changes to their entities. I generate a migration for these changes. But I have to do manual clean up on them because the migrations include removing tables for entities that are in another branch! This problem would not exist if the schema diff was against the migrations and not the current schema. To work around this we have implemented a bit of a hack with a script that first applies all existing migrations to a completely empty database and then we create new migrations on that database. This is essentially using the current migrations to do the diff. It would be nice if this was either not necessary OR if this was a built-in option of the migration system. Thanks! |
We can't make diffs based on migrations, migrations do not bare metadata, only SQL queries, they would first need to be executed somewhere. Also you say "against migrations", but migrations do not necessarily have to be "complete", e.g. you could start using them on existing database. Might be also performance heavy if we do use actual database, if you reach certain number of existing migrations (and I'd guess it would be rather small number). But maybe we could find a way in the middle. The new schema diffing in v5 already works based on comparing metadata, so we could serialize the whole database state into some migration meta file. There would be just one file, if not found, we would use the current db state. If we create new migration, the file would be updated. It would be versioned, so if you check out different branch, you'd have the state from that branch. Will try to hack something together, doing it this way might be actually quite easy. |
That sounds like a clever solution. |
A bit harder than I expected, but looks like it's working (#1815). If you are brave enough, it will be available in |
Would be a nice feature if there is a way to pass metadata from the save/insert/... method to a hook. In example the user who performed the operation. |
I already have reference/collection loading working in v3, as well as find/findOne (even though I don't support all the operators, just a subset). I've just migrated my codebase from v3 to v4 and I'm in the process of forward porting batched selects as well, but since it's a lot of work I've decided to skip v4 and target v5 directly. I'm not sure if I will ever be able to target the whole set of operators, but maybe this the right time to try to get it merged. I'll update my batched selects to v5 while also updating the PR to get rid of gql altogether and make it easier to review. |
I've rebased my dataloader to v5, if someone wants to have a look you can find it here: #364 (comment) |
@WumaCoder I ended up creating my own cache layer (using subscribers to update the cache). It would be a bit too complicated for MikroORM to auto-update the cache. Consider there might also be fields related to other tables etc. But if it's feasable, this would be a super powerful feature. |
I am considering dropping support for node 12/13 in v5. Given the stable release will probably happen early next year and the EOL of node 12 is 2022-04-30, it feels like it does not matter much to keep supporting it. Similarly we might require TS 4.5 (we will definitely adopt it right after it gets released, the question is whether there will be actual BCs like they did in 4.1 with string literal types). There is no particular need for this right now, but given how long the v5 development took, I'd rather be more aggressive now, so we don't fall into "not possible to upgrade dependency A because it requires node 14+ and we keep supporting node 12" (or similarly with TS, e.g. knex required TS 4.1 in latest versions, so 4.x could not use it and the upgrade will need to wait for v5). Anybody still stuck on node <14? Or anybody that would see upgrading of TS as problematic? I remember there were people stuck on some particular version of TS due to angular not supporting newer ones, etc. (btw I also made quite a lot of progress in last week or two, if you are using v5 already, please try the latest changes) |
I'm all in for node 14+, but I'm currently stuck with TS 4.4 because of #1623 (comment) |
So TS 4.5 is finally out, latest master is updated and also used in the realworld project, everything works fine there. https://github.com/mikro-orm/nestjs-realworld-example-app/tree/v5 edit: also tried to run it with TS 4.1 and everything looks fine |
Is there an ETA for this release as it appears that everything has been checked? Great work. |
Yeah most of the things are ready, I'd like to ship RC before christmas, and stable probably in january if there are no major issues found. Will be also shipping one more patch release to 4.x branch, with some of the bug fixes that were easily portable. |
I am using v5 on daily basis and am very happy so far! |
How did you install it? |
just |
Added two more breaking changes to v5:
|
I'm so excited for the strictly typed |
any eta on v5 |
Two more small BCs done via #2605
|
One final breaking change via #2660 before I ship the RC: Population no longer infers the where condition by default
Previously when we used populate hints in // this would end up with `Author.books` collections having only books of PK 1, 2, 3
const a = await em.find(Author, { books: [1, 2, 3] }, { populate: ['books'] }); Following this example, if we wanted to load all books, we would need a separate const a = await em.find(Author, { books: [1, 2, 3] });
await em.populate(a, ['books']); This behaviour changed and is now configurable both globally and locally, via // revert back to the old behaviour locally
const a = await em.find(Author, { books: [1, 2, 3] }, {
populate: ['books'],
populateWhere: PopulateHint.INFER,
}); |
So here we go, first RC version is officially out! Here is the full changelog: https://github.com/mikro-orm/mikro-orm/blob/master/CHANGELOG.md#500-rc0-2022-01-23 |
@B4nan Awesome to see the RC being released! Unless there are any major issues being reported, how soon can we expect the final release to drop? |
I just want to polish the docs a bit (mainly fix things that are no longer valid), and write the release article. So codewise I don't plan anything, only bug fixes, and maybe #2677 as it seems a bit breaking to me, so would be better to deal with it in 5.0.0. But writing docs and articles is kinda the least entertaining part of this, so I tend to postpone it :D I hope it won't be more than a week or two. |
I would definitely love to see it merged in 5.0.0. |
Hi! I've tested v5-rc0 in our codebase at Qerko and everything works correctly - the pipeline is green. Currently, we use just basic mikro-orm functionality so I didn't expect any major issues. But I think it's good to let you know that everything looks good from our side. |
I've been thinking about how to preserve optionality in the Following entity would require only @Entity()
export class Publisher {
// union of keys that are defined as required but are in fact optional
[OptionalProps]?: 'type';
@PrimaryKey()
id!: number;
@Property()
name!: string;
@OneToMany({ entity: () => Book, mappedBy: 'publisher' })
books = new Collection<Book>(this);
@ManyToMany({ entity: () => Test, eager: true })
tests = new Collection<Test>(this);
@Enum(() => PublisherType)
type = PublisherType.LOCAL;
} Do you think it would be worthy addition? |
So the above is now implemented, as well as runtime checks for required properties when flushing new entities. Worth noting this will now effectively force mongo users to specify optionallity via Will ship this as RC.1 later today (already available in |
Regarding the new @Entity()
export class User {
[OptionalProps]?: 'createdAt' | 'updatedAt';
@PrimaryKey({ columnType: 'uuid', defaultRaw: 'gen_random_uuid()' })
id: string = uuidv4();
@Property({ defaultRaw: 'CURRENT_TIMESTAMP' })
createdAt: Date = new Date();
@Property({ defaultRaw: 'CURRENT_TIMESTAMP', onUpdate: () => new Date() })
updatedAt: Date = new Date();
@Property({ nullable: false })
firstName: string;
@Property({ nullable: false })
lastName: string;
get fullName(): string {
return `${this.firstName} ${this.lastName}`;
}
} It totally makes sense to include If not, it's still a great feature! So thanks anyhow ❤️ |
I don't think so, a getter is still a property. Technically you could have a regular (and required) property defined with get/set too, it is not always virtual. We don't have any information from decorators on type level either. Obviously if anybody knows a trick to achieve this, I am all ears :] All I can think of is kind of a prisma style generated interfaces, so we can actually work with the decorator metadata. I would very much like to allow that (optionally) at some point. |
RC.3 is out, with some last minute changes, namely validation improvements If there won't be any major issue, I will probably ship the stable tomorrow. https://github.com/mikro-orm/mikro-orm/blob/master/CHANGELOG.md#500-rc3-2022-02-05 |
Time has come! Say hello to MikroORM 5! https://medium.com/@b4nan/mikro-orm-5-stricter-safer-smarter-b8412e84cca4 |
Progress
{ type: t.decimal }
)numeric(10,2)
orvarchar(100)
)AsyncLocalStorage
in theRequestContext
helpergen-esm-wrapper
(ECMAScript Modules support #1010)em.create()
(Strict typing forem.create
#1456)toObject()
,toJSON()
andtoPOJO()
return typesskip locked/nowait
) (Postgres SKIP LOCKED #1786)FindOptions.orderBy
, allowing arrays of objects too (Order of multiple nested orderBy params #2010)e0/1/2/3...
) and make it configurable (feat(core): allow configuring aliasing naming strategy #2419)QueryBuilder
instance (feat(query-builder): allow awaiting theQueryBuilder
instance #2446)Logger
instance (feat(core): allow providing customLogger
instance #2443)em.populate()
(EntityManager.populate
ignores lazy scalar properties. #1479)Promise.all()
(Support running EntityManager inside Promise.all #2412)Breaking changes
em.create()
,em.assign()
,qb.insert()
,qb.update()
,e.toObject()
,e.toJSON()
, ...)FindOptions.orderBy/fields/
are now strictly typedfind/findOne/...
parameters (populate/orderBy/limit/offset
) in favour ofFindOptions
em.fork()
@Repository
decorator (use@Entity({ customRepository: () => CustomRepository })
instead)populateAfterFlush
by defaultmigrations.pattern
is removed in favour ofmigrations.glob
(chore: upgrade umzug to 3.0 #2556)https://mikro-orm.io/docs/next/upgrading-v4-to-v5/
mikro-orm/nestjs-realworld-example-app#36
The text was updated successfully, but these errors were encountered: