New Scraper and NN #1387
Merged
New Scraper and NN #1387
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Needs to be test built for Windows.
NeuralNet interface
Also add a check for wallet sync at the start of scraper and NN thread init. Fix Windows NN compilation with new NN interface.
…to integrated_scraper This should fix the zlib compilation problems
This is part 1 of hooking the scraper into the new NN interface. There are first draft implementations of ScraperGetNeuralContract ScraperGetNeuralHash and a stub for ScraperSynchronizeDPOR I restructured some of the code and implemented a second lock in preparation for the ScraperSynchronizeDPOR call.
Needs to be test built for Windows.
Fix link errors
Initial version of ScraperSynchronizeDPOR. Implementation of single shot mode through Scraper() to support on command stats updates via ScraperSynchronizeDPOR.
This changes the block granularity (mod) in GetConsensusBeaconList() to 4 * BLOCK_GRANULARITY or roughly one hour, to improve manifest consensus.
Stub out to true instead of false for testing.
Stub out to true instead of false for testing.
The global stats cache prevents repeated updating of the stats on calls of the neural contract. ExplainMagnitude is only partially completed.
…ridcoin-Research into integrated_scraper
Implmement local node check as first option in rpc call before going to network.
This commit makes a few small adjustments to ensure the new NN is fully enabled when selected, and also that when running on Windows, the old NN is disabled if the new NN is selected.
This was causing an occasional segmentation fault.
This greatly cleans up the GUI code as it no longer needs to track the QAxObject instantiation. It also isolates the old NN implementation, making it easier to replace.
We cannot keep it in neuralnet_win32.cpp since it gets linked into an archive and is discarded during link time.
Isolate old NN
Change sleep interval defaults in init.cpp.
|
This is a comment to link issues to this PR... |
Be explicit about build dependency on pkg-config
…emoval_of_01234 Removal of 0123 0124 & implementation of MinedType GetGeneratedType - II
|
Ok, this is now in sync with my other tree, integrated_scraper_removalof01230124 and I am shifting back to the integrated_scraper tree, since PR #1390 was merged. |
jamescowens
force-pushed
the
integrated_scraper
branch
6 times, most recently
from
8bb9f8c
to
4df23ed
Compare
This commit implements a simple log archiver for the scraper log. If people like this I will do something similar for the main log as well. Currently it runs in the housekeeping loop and automatically renames the scraper.log to scraper-YYYYMMDDHHMMSS.log then compresses the archive log to a gz file. A lock is held only during the rename, and a new scraper.log is started immediately. Stream compression (just like the rest of the scraper) is used to minimize memory consumption. I have also provided an rpc command, archivescraperlog, which takes no arguments and can be used to immediately roll the current log to an archive file and start a new one non-disruptively.
jamescowens
force-pushed
the
integrated_scraper
branch
from
4df23ed
to
b06101c
Compare
of GetDataDir() providing the proper value.
whether the scraper rolls the scraper.log to archive at midnight UTC every day or not. The default is true. Also did some other minor cleanups.
to correct compilation problem for 32-bit Windows.
…ridcoin-Research into integrated_scraper
denravonska
approved these changes
Feb 25, 2019
Use native SSL stack in scraper for Windows builds
denravonska
added a commit
that referenced
this pull request
May 10, 2019
Added: - Replace NeuralNetwork with portable C++ scraper #1387 (@jamescowens, @tomasbrod, @Cycy, @TheCharlatan, @denravonska). - Allow compile flags to be used for depends #1423 (@G-UK). - Add stake splitting and side staking info to getmininginfo #1424 (@jamescowens). - Add freedesktop.org desktop file and icon set #1438 (@a123b). Changed: - Disable Qt for windows Travis builds #1276 (@TheCharlatan). - Replace use of AppCache PROJECT section with strongly-typed structures #1415 (@cyrossignol). - Change dumpwallet to use appropriate data directory #1416 (@jamescowens). - Optimize ExtractXML() calls by avoiding unnecessary string copies #1419 (@cyrossignol). - Change signature of IsLockTimeWithinMinutes #1422 (@jamescowens). - Restore old poll output for getmininginfo RPC #1437 (@a123b). - Prevent segfault when using rpc savescraperfilemanifest #1439 (@jamescowens). - Improve miner status messages for ineligible staking balances #1447 (@cyrossignol). - Enhance scraper log archiving #1449 (@jamescowens). Fixed: - Re-enable full GUI 32-bit Windows builds - part of #1387 (@jamescowens). - Re-activate Windows Installer #1409 (@TheCharlatan). - Fix Depends and Travis build issues for ARM #1417 (@jamescowens). - Fix syncupdate icons #1421 (@jamescowens). - Fix potential BOINC crash when reading projects #1426 (@cyrossignol). - Fix freeze when unlocking wallet #1428 (@denravonska). - Fix RPC after high priority alert #1432 (@denravonska). - Fix missing poll in GUI when most recent poll expired #1455 (@cyrossignol). Removed: - Remove old, rudimentary side staking implementation #1381 (@denravonska). - Remove auto unlock #1402 (@denravonska). - Remove superblock forwarding #1430 (@denravonska).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is the new scraper and "neural network".