Repository navigation
Importing Data
FAuction can import the data of another auction house plugin: the items on sale, the items waiting to be collected, the sales history and the money still owed to players. This page is for server administrators; developers who want to write an import module for another plugin should read Importer API.
-
A shared SQL database: MySQL, MariaDB or PostgreSQL (see Configuration → database.yml). The import refuses to run in SQLite mode: in SQLite, FAuction keeps the auctions in memory and rewrites the
auctionstable from that memory when the server stops, so anything written to the database by an import would be lost. Move FAuction to one of the SQL databases first, then import. - The source plugin disabled. Every supported source keeps its data in memory and writes it back to its files or database: while it runs, it would still hand out the very items and money you just imported (duplication), and what you import could be stale. The modules refuse to run while the source plugin is enabled.
- The permission
fauction.admin.import(orfauction.admin). All the import commands work from the console, which is the recommended way.
- Back up the FAuction database. An import writes thousands of rows that cannot be told apart from the others afterwards.
-
Stop the server, remove the jar of the source plugin (keep its folder in
plugins/: that is what is read), and start the server again. Ideally keep the server in maintenance (whitelist) until the end. -
/ah admin import list— shows the available modules, the types of data each one imports, and whether it found the data of its source. -
/ah admin import run <id> --dry-run— reads everything, validates everything, writes nothing. Read the summary and the detailed report (see Reading the report). -
/ah admin import run <id>— FAuction asks for a confirmation: type/ah admin import run <id> confirmwithin 30 seconds. The import runs with the options of the first command. Follow it with/ah admin import status. - Check in game:
/ah,/ah expire, the history. Money owed to players is paid the next time they are online (the same mechanism as a sale to an offline seller). - Delete the folder of the source plugin once you are satisfied (or keep it as an archive: running the import again never duplicates anything, see Running an import again).
| Command | Description |
|---|---|
/ah admin import list |
The registered modules, their supported types and whether they can run now. |
/ah admin import run <id> [options] |
Runs an import. Without --dry-run, it has to be confirmed with /ah admin import run <id> confirm within 30 seconds, by the same player (or the console). |
/ah admin import status |
Progress of the running import (rows read, imported, already imported, refused). With nothing running, shows the summary of the last one. |
/ah admin import cancel |
Stops the running import cleanly: the batch being written is finished, nothing after it. Running the import again later picks up where it stopped. |
Only one import runs at a time. The import runs off the main thread; the server stays responsive.
| Option | Description |
|---|---|
--dry-run |
Validate and report, write nothing. No confirmation needed. |
--types auction,expired,historic,pending_currency |
Import only these types (pending works too). Default: every type the module supports. |
--expired-to-expires |
A sale that is already expired by FAuction's expiration.time goes straight to the expired items of its seller. Without it, it is put on sale and FAuction's own expiration moves it at its next check. |
--apply-limits |
A sale that breaks the price limits (min-price, max-price, min-price-default, max-price-default) or the item-blacklist of config.yml is given back to its seller (expired items) instead of being put on sale. Not applied by default: you are taking over existing sales. The number of sales per rank (limitations) is never applied to an import. |
--other-currency=reject|expire |
A sale in another currency than FAuction's currencyUse (see Currencies) is refused (reject, default) or given back to its seller (expire). A refused sale is listed in the report and can be imported later with expire. |
--bids=convert|skip |
Auctions with bids of the source: converted (default, see Auctions with bids) or skipped and listed in the report. |
--force-reimport |
Ignore the journal of the previous imports and import everything again. Duplicates what was already imported: only for an import whose rows you deleted by hand. |
--opt key=value |
An option of the module itself (repeatable), see the modules below. |
Options take --name value or --name=value. Example: /ah admin import run auctionhouse-elaine --types auction,expired --opt apply-source-tax=false --dry-run.
At the end, FAuction prints a summary per data type:
- read: rows the module produced;
- imported: rows written (of which given back to the seller: sales moved to the expired items by an option);
- already imported: rows a previous import brought, ignored;
- rejected: rows refused by FAuction's validation (invalid price, unreadable item, air, unmapped currency...);
- failed: rows of a batch the database refused (the batch was rolled back, nothing of it was written; run the import again).
Plus the rows the module skipped itself (deleted by an administrator in the source, auction with bids with --bids=skip...).
The detailed report is written to plugins/FAuction/imports/<id>-<date>.log: the counters, then every refused, skipped or failed row with its source identifier and the reason. Nothing read from the source disappears without being either imported or listed there.
Every imported row is recorded in the fa_import_log table (module, type, identifier of the row in the source), in the same transaction as the row itself. Running the same import again therefore only imports what was not imported yet:
- after a crash or a
cancel, the next run picks up exactly where the previous one stopped; - after fixing a refused row (adding a currency to
currency-map, for instance), the next run only imports that row.
The modules evaluate the state of each sale (on the market, expired, auction running or ended) at the moment the source plugin last saved its files, that is when it stopped, not at the moment of the import. Two runs a week apart therefore see exactly the same states, and a sale is never imported once as on sale and once as expired. A sale that was on the market when the source stopped is put on sale, and FAuction's own expiration (expiration.time) takes over from its listing date.
When the database refuses a batch, its rows are retried one by one: only the rows really refused are reported as failed. Items larger than the item column (65,535 bytes on MySQL/MariaDB, e.g. a shulker box full of written books) and stacks of more than 99 items are refused by the validation and listed in the report.
--force-reimport asks for confirmation with an explicit warning: it ignores the journal, so everything already imported is imported again — items and money twice — unless you deleted those rows by hand.
The money owed by a source, and the sales of sources with several currencies, carry the name of their currency on the source side. FAuction converts it with import.currency-map of config.yml (see Configuration → import):
import:
currency-map:
Vault: VAULT
Experience: EXPERIENCE
Level: LEVELA currency missing from that map is never converted at random: the row is refused and listed in the report. Add the currency to the map (CoinsEngine-coins: VAULT...) only if the two currencies really are worth the same, then run the import again.
FAuction has no bids yet. When a source has auctions with bids whose money is held by the source (escrow), each one is converted so that no item and no money is lost:
| State in the source | Imported as |
|---|---|
| Running, no bid | the item goes back to the seller (expired items) |
| Running, with bids | the item goes back to the seller + every bidder is refunded of their last bid (money owed) |
| Ended, winner has not collected the item | the item goes to the winner's expired items |
| Ended, seller has not collected the money | the final price is owed to the seller |
| Ended, losers not refunded yet | each of them is refunded of their last bid |
With --bids=skip, those auctions are not imported at all and are listed in the report.
- Tested with Auction-House 1.5.6.
- Reads
plugins/AuctionHouse/data/notes.json(orplugins/AuctionHouse/notes.jsonfor older versions), plusconfig.yml(tax,auction-setup-time,bin-auction-duration,bid-auction-duration) andlayout.ymlof the source. - Currency: Vault (
Vaultincurrency-map).
| In the source | Imported as |
|---|---|
| Sale on the market | sale (auction) |
| Sale partially bought | the rest stays on sale at the same unit price; the part bought is money owed to the seller + history |
| Expired, or expired by an administrator | expired item of the seller |
| Deleted by an administrator | skipped: the source replaced the item by its "deleted" dirt item, the original no longer exists |
| Sold, money not collected by the seller | money owed to the seller (net of the source's tax) + history (without the date of sale, which the source does not keep) |
| Auctions with bids | see Auctions with bids |
Options: --opt apply-source-tax=false pays the sellers the full price instead of the price minus the source's tax.
Known limits: the source's text logs (logs/*.log) are not imported (no UUIDs); a sale sold before the source stored the buyer's UUID gets no history line (the money is still owed, the line is listed in the report). An auction with bids expired by an administrator is handled like an ended one (winner served, seller paid): check those in the report.
- Tested with NexusAuctionHouse 2.4.7.
- Reads
plugins/NexusAuctionHouse/data/bins.jsonanddata/expired-retrieve.json, plusconfig.yml(expiry.enable,expiry.time). The old.ymldata files are not read: start NexusAuctionHouse once, it converts them itself. - No money is ever pending: the source pays the seller at once, even offline.
| In the source | Imported as |
|---|---|
| Sale on the market | sale, listing date deduced from expiry.time
|
Sale while expiry.enable: false
|
sale |
| Expired sale |
not imported from bins.json: its item is already in the seller's expired-retrieve.json list, imported below (importing both would duplicate it) |
| Bought sale | history only (seller already paid, item already delivered) |
| Withdrawn by its seller | skipped (listed in the report) |
Each item of expired-retrieve.json
|
expired item of that player |
Options: --opt recover-orphans=true gives back to the seller an expired sale whose item is not found in the seller's list. The source loses those items when a sale expires while the server is stopped; the option recovers them. But the file cannot tell a lost item from one the seller already collected: every expired sale collected during the last log-keep-time is given a second time. Only use it when you know items were lost, and check the report.
Known limits: the history only covers the source's log-keep-time (30 days by default); the global money statistic of bins.json is ignored.
Planned as a separate plugin, FAuction-Import-AxAuctions, not available yet: AxAuctions is closed source and its database schema has to be checked on a real server first.
"The import needs a shared SQL database" — you are in SQLite mode: see Requirements.
"... is still enabled" — remove the jar of the source plugin and restart (step 2).
A row is "rejected: Currency X is not in import.currency-map" — add X: VAULT (or another FAuction currency) to import.currency-map if they are worth the same, /ah admin reload, and run the import again: only the refused rows are imported.
A row is "rejected: Item cannot be decoded" — the item was saved by a Minecraft version or a plugin this server cannot read. It is listed in the report so you can give it back by hand.
Several servers share the database — run the import on one server only. The others see the imported data at their next cache update (cacheUpdate). If two servers ran the same import at the same time anyway, the unique constraint of fa_import_log would refuse the second copy of every row.