New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Architecture notes / DataModel description #1138

Closed
mwjames opened this Issue Sep 4, 2015 · 4 comments

Comments

3 participants
@mwjames
Contributor

mwjames commented Sep 4, 2015

I don't know where to put these (created as part of an issue investigation) but to avoid having them lost in the realm of the issues, I'll use this as place holder so interested parties can have a read.

MediaWiki dependencies

smw-mw-interface

Store

image

image

UpdateJob

image

ParserData

image

EmbeddedQueryDependencyLinksStore

image

image

DeferredRequestDispatchManager

image

@JeroenDeDauw

This comment has been minimized.

Show comment
Hide comment
Member

JeroenDeDauw commented Sep 27, 2015

@WolfgangFahl

This comment has been minimized.

Show comment
Hide comment
@WolfgangFahl

WolfgangFahl Jan 2, 2016

Contributor

The images are all uploaded and embedded in the Architecture guide now.
The issue may IMHO be closed since #1343 is still open.

Contributor

WolfgangFahl commented Jan 2, 2016

The images are all uploaded and embedded in the Architecture guide now.
The issue may IMHO be closed since #1343 is still open.

@mwjames

This comment has been minimized.

Show comment
Hide comment
@mwjames

mwjames Mar 12, 2016

Contributor

Most pictures where copied from here to the wiki and I may post an update on this in future but for now this is completed from where I'm standing.

Contributor

mwjames commented Mar 12, 2016

Most pictures where copied from here to the wiki and I may post an update on this in future but for now this is completed from where I'm standing.

@mwjames mwjames closed this Mar 12, 2016

@mwjames

This comment has been minimized.

Show comment
Hide comment
@mwjames

mwjames Mar 22, 2016

Contributor

InMemoryPoolCache

Just to give some idea about our InMemoryPoolCache (which uses the FixedInMemoryLruCache to ensure an array stays fixed and doesn't inflate during run time with a Lru [0] algo to only keep the most used members in the cache space) to help us process things a bit quicker.

Running rebuildData.php on a 100 ID sample (which before would have accounted for a direct Store requests) with 2.4.

Processing all IDs from 1 to 100 ...

............................................................ 61%
........................................

100 IDs refreshed.

property.spec.lookup.cache          inserts : 195   deletes : 0 max : 500 count : 195 hits : 41775  misses : 0
exporter.dataitem.resource.encoder  inserts : 1425  deletes : 0 max : 500 count : 500 hits : 19938  misses : 0
sql.store.dataitem.finder           inserts : 1298  deletes : 0 max : 500 count : 500 hits : 141    misses : 0
sql.store.redirect.infostore        inserts : 132   deletes : 0 max : 500 count : 132 hits : 5176   misses : 0
sparql.store.redirectlookup         inserts : 166   deletes : 0 max : 500 count : 166 hits : 11324  misses : 0
store.redirectTarget.lookup         inserts : 1     deletes : 1 max : 500 count : 0   hits : 1      misses : 0

[0] http://mcicpc.cs.atu.edu/archives/2012/mcpc2012/lru/lru.html

Contributor

mwjames commented Mar 22, 2016

InMemoryPoolCache

Just to give some idea about our InMemoryPoolCache (which uses the FixedInMemoryLruCache to ensure an array stays fixed and doesn't inflate during run time with a Lru [0] algo to only keep the most used members in the cache space) to help us process things a bit quicker.

Running rebuildData.php on a 100 ID sample (which before would have accounted for a direct Store requests) with 2.4.

Processing all IDs from 1 to 100 ...

............................................................ 61%
........................................

100 IDs refreshed.

property.spec.lookup.cache          inserts : 195   deletes : 0 max : 500 count : 195 hits : 41775  misses : 0
exporter.dataitem.resource.encoder  inserts : 1425  deletes : 0 max : 500 count : 500 hits : 19938  misses : 0
sql.store.dataitem.finder           inserts : 1298  deletes : 0 max : 500 count : 500 hits : 141    misses : 0
sql.store.redirect.infostore        inserts : 132   deletes : 0 max : 500 count : 132 hits : 5176   misses : 0
sparql.store.redirectlookup         inserts : 166   deletes : 0 max : 500 count : 166 hits : 11324  misses : 0
store.redirectTarget.lookup         inserts : 1     deletes : 1 max : 500 count : 0   hits : 1      misses : 0

[0] http://mcicpc.cs.atu.edu/archives/2012/mcpc2012/lru/lru.html

@kghbln kghbln added this to Documentation in Documentation Apr 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment