Skip to content

Commit

Permalink
Update changelog for koios-1.0.6
Browse files Browse the repository at this point in the history
  • Loading branch information
@rdlrt
rdlrt committed Aug 12, 2022
1 parent a031f6b commit 610fb28
Show file tree
Hide file tree
Showing 10 changed files with 563 additions and 211 deletions.
10 changes: 5 additions & 5 deletions Appendix/postgres/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -977,13 +977,13 @@ <h4 id="verify-login-to-postgres-instance">Verify Login to postgres instance<a c
<span class="nb">echo</span> <span class="s2">&quot;/var/run/postgresql:5432:cexplorer:*:*&quot;</span> &gt; <span class="nv">$PGPASSFILE</span>
chmod <span class="m">0600</span> <span class="nv">$PGPASSFILE</span>
psql postgres
<span class="c1"># psql (13.4)</span>
<span class="c1"># psql (14.0)</span>
<span class="c1"># Type &quot;help&quot; for help.</span>
<span class="c1"># </span>
<span class="c1"># postgres=#</span>
</code></pre></div>
<h4 id="tuning-your-instance">Tuning your instance<a class="headerlink" href="#tuning-your-instance" title="Permanent link">⚓︎</a></h4>
<p>Before you start populating your DB instance using dbsync data, now might be a good time to put some thought on to baseline configuration of your postgres instance by editing <code>/etc/postgresql/13/main/postgresql.conf</code>.
<p>Before you start populating your DB instance using dbsync data, now might be a good time to put some thought on to baseline configuration of your postgres instance by editing <code>/etc/postgresql/14/main/postgresql.conf</code>.
Typically, you might find a lot of common standard practices parameters available in tuning guides. For our consideration, it would be nice to start with some baselines - for which we will use inputs from example <a href="https://pgtune.leopard.in.ua/#/">here</a>.
You might want to fill in some sample information as per below to fill in the form:</p>
<table>
Expand All @@ -996,7 +996,7 @@ <h4 id="tuning-your-instance">Tuning your instance<a class="headerlink" href="#t
<tbody>
<tr>
<td>DB Version</td>
<td>13</td>
<td>14</td>
</tr>
<tr>
<td>OS Type</td>
Expand All @@ -1008,7 +1008,7 @@ <h4 id="tuning-your-instance">Tuning your instance<a class="headerlink" href="#t
</tr>
<tr>
<td>Total RAM</td>
<td>32 (or as per your server)</td>
<td>64 (or as per your server)</td>
</tr>
<tr>
<td>Number of CPUs</td>
Expand All @@ -1024,7 +1024,7 @@ <h4 id="tuning-your-instance">Tuning your instance<a class="headerlink" href="#t
</tr>
</tbody>
</table>
<p>In addition to above, due to the nature of usage by dbsync (restart of instance does a rollback to start of epoch), and data retention on blockchain - we're not affected by loss of volatile information upon a restart of instance. Thus, we can relax some of the data retention and protection against corruption related settings, as those are IOPs/CPU Load Average impacts that the instance does not need to spend. We'd recommend setting 3 of those below in your <code>/etc/postgresql/13/main/postgresql.conf</code>:</p>
<p>In addition to above, due to the nature of usage by dbsync (restart of instance does a rollback to start of epoch), and data retention on blockchain - we're not affected by loss of volatile information upon a restart of instance. Thus, we can relax some of the data retention and protection against corruption related settings, as those are IOPs/CPU Load Average impacts that the instance does not need to spend. We'd recommend setting 3 of those below in your <code>/etc/postgresql/14/main/postgresql.conf</code>:</p>
<table>
<thead>
<tr>
Expand Down
233 changes: 224 additions & 9 deletions Build/grest-changelog/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,86 @@
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>

<li class="md-nav__item">
<a href="#106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13" class="md-nav__link">
[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13
</a>

<nav class="md-nav" aria-label="[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes" class="md-nav__link">
Data Output Changes
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers" class="md-nav__link">
Changes for Instance Providers
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#105-alpha-networks-only" class="md-nav__link">
[1.0.5] - alpha networks only
</a>

<nav class="md-nav" aria-label="[1.0.5] - alpha networks only">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api_1" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes_1" class="md-nav__link">
Data Output Changes
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers_1" class="md-nav__link">
Changes for Instance Providers
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#101" class="md-nav__link">
[1.0.1]
Expand All @@ -429,15 +509,15 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api" class="md-nav__link">
<a href="#changes-for-api_2" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes" class="md-nav__link">
<a href="#data-output-changes_2" class="md-nav__link">
Data Output Changes
</a>

Expand All @@ -456,7 +536,7 @@
</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers" class="md-nav__link">
<a href="#changes-for-instance-providers_2" class="md-nav__link">
Changes for Instance Providers
</a>

Expand Down Expand Up @@ -949,6 +1029,86 @@
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>

<li class="md-nav__item">
<a href="#106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13" class="md-nav__link">
[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13
</a>

<nav class="md-nav" aria-label="[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes" class="md-nav__link">
Data Output Changes
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers" class="md-nav__link">
Changes for Instance Providers
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#105-alpha-networks-only" class="md-nav__link">
[1.0.5] - alpha networks only
</a>

<nav class="md-nav" aria-label="[1.0.5] - alpha networks only">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api_1" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes_1" class="md-nav__link">
Data Output Changes
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers_1" class="md-nav__link">
Changes for Instance Providers
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#101" class="md-nav__link">
[1.0.1]
Expand All @@ -972,15 +1132,15 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#changes-for-api" class="md-nav__link">
<a href="#changes-for-api_2" class="md-nav__link">
Changes for API
</a>

<nav class="md-nav" aria-label="Changes for API">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#data-output-changes" class="md-nav__link">
<a href="#data-output-changes_2" class="md-nav__link">
Data Output Changes
</a>

Expand All @@ -999,7 +1159,7 @@
</li>

<li class="md-nav__item">
<a href="#changes-for-instance-providers" class="md-nav__link">
<a href="#changes-for-instance-providers_2" class="md-nav__link">
Changes for Instance Providers
</a>

Expand Down Expand Up @@ -1056,6 +1216,61 @@


<h1 id="koios-grest-changelog">Koios gRest Changelog<a class="headerlink" href="#koios-grest-changelog" title="Permanent link">⚓︎</a></h1>
<h2 id="106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13">[1.0.6/1.0.6m] - Interim release for all networks to upgrade to dbsync v13<a class="headerlink" href="#106106m-interim-release-for-all-networks-to-upgrade-to-dbsync-v13" title="Permanent link">⚓︎</a></h2>
<p>The backlog of items not being added to mainnet has been increasing due to delays with Vasil HFC event to Mainnet. As such we had to come up with a split update approach.
The mainnet nodes are still not qualified to be Vasil-ready (in our opinion) for 1.35.x , but dbsync 13 can be used against node 1.34.1 fine. In order to cater for this split, we have added an intermediate koios-1.0.6m tag that brings dbsync updates while maintaining node-1.34.1.</p>
<h3 id="changes-for-api">Changes for API<a class="headerlink" href="#changes-for-api" title="Permanent link">⚓︎</a></h3>
<h4 id="data-output-changes">Data Output Changes<a class="headerlink" href="#data-output-changes" title="Permanent link">⚓︎</a></h4>
<ul>
<li><code>pool_delegators</code> - <code>epoch_no</code> =&gt; <code>active_epoch_no</code> <a href="https://github.com/cardano-community/guild-operators/pull/1454">#1454</a></li>
<li><code>asset_history</code> - Add <code>block_time</code> and <code>metadata</code> fields for all previous mint transactions <a href="https://github.com/cardano-community/guild-operators/pull/1468">#1468</a></li>
<li><code>asset_info</code> - Retain latest mint transaction instead of first (in line with most CIPs as well as pool metadata - latest valid meta being live) <a href="https://github.com/cardano-community/guild-operators/pull/1468">#1468</a></li>
<li>Ensure all output date formats is integer to keep in line with UNIX timestamps - to be revised in future if/when there are milliseconds <a href="https://github.com/cardano-community/guild-operators/pull/1460">#1460</a></li>
<li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li>
<li><code>/genesis</code> =&gt; <code>systemStart</code></li>
<li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li>
<li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li>
<li><code>/asset_info</code> =&gt; <code>creation_time</code></li>
<li><code>tx_info</code> - Add Vasil data <a href="https://github.com/cardano-community/guild-operators/pull/1464">#1464</a></li>
<li><code>collaterals</code> =&gt; <code>collateral_inputs</code></li>
<li>Add <code>collateral_outputs</code>, <code>reference_inputs</code> to <code>tx_info</code></li>
<li>Add <code>datum_hash</code>, <code>inline_datum</code>, <code>reference_script</code> to collateral input/outputs, reference inputs &amp; inputs/outputs JSON.</li>
<li>Add complete <code>cost_model</code> instead of <code>cost_model_id</code> reference</li>
<li><code>epoch_params</code> - Update leftover lovelace references to text for consistency: <a href="https://github.com/cardano-community/guild-operators/pull/1484">#1484</a></li>
<li><code>key_deposit</code></li>
<li><code>pool_deposit</code></li>
<li><code>min_utxo_value</code></li>
<li><code>min_pool_cost</code></li>
<li><code>coins_per_utxo_size</code></li>
</ul>
<h3 id="changes-for-instance-providers">Changes for Instance Providers<a class="headerlink" href="#changes-for-instance-providers" title="Permanent link">⚓︎</a></h3>
<ul>
<li><code>get-metrics.sh</code> - Add active/idle connections to database <a href="https://github.com/cardano-community/guild-operators/pull/1459">#1459</a></li>
<li><code>grest-poll.sh</code>: Bump haproxy to 2.6.1 and set default value of API_STRUCT_DEFINITION to be dependent on network used. <a href="https://github.com/cardano-community/guild-operators/pull/1450">#1450</a></li>
<li>Lighten <code>grest.account_active_stake_cache</code> - optimise code and delete historical view (beyond 4 epochs). [<a class="magiclink magiclink-github magiclink-issue" href="https://github.com/squidfunk/mkdocs-material/issues/1451" title="GitHub Issue: squidfunk/mkdocs-material #1451">#1451</a>(<a href="https://github.com/cardano-community/guild-operators/pull/1451">https://github.com/cardano-community/guild-operators/pull/1451</a>)</li>
<li><code>tx_metalabels</code> - Move metalabels from view to RPC using lose indexscan (much better performance) <a href="https://github.com/cardano-community/guild-operators/pull/1474">#1474</a></li>
<li>Major re-work to artificially add last epoch's active stake cache data (brings in latest snapshot information without depending on node), not used in endpoints for this release <a href="https://github.com/cardano-community/guild-operators/pull/1452">#1452</a></li>
<li><code>grest.stake_snapshot_cache</code> - Fix rewards for new accounts <a href="https://github.com/cardano-community/guild-operators/pull/1476">#1476</a></li>
</ul>
<h2 id="105-alpha-networks-only">[1.0.5] - alpha networks only<a class="headerlink" href="#105-alpha-networks-only" title="Permanent link">⚓︎</a></h2>
<p>Since there have been a few deviations wrt Vasil for testnet and mainnet, this version only targets networks except Mainnet!</p>
<h3 id="changes-for-api_1">Changes for API<a class="headerlink" href="#changes-for-api_1" title="Permanent link">⚓︎</a></h3>
<h4 id="data-output-changes_1">Data Output Changes<a class="headerlink" href="#data-output-changes_1" title="Permanent link">⚓︎</a></h4>
<ul>
<li><code>/epoch_info</code> - Add <code>total_rewards</code> and <code>avg_block_reward</code> for a given epoch <a href="https://github.com/cardano-community/koios-artifacts/pull/43">#43</a></li>
<li>Update all date output formats to return UNIX timestamp (as per poll held in discussions group): <a href="https://github.com/cardano-community/koios-artifacts/pull/45">#45</a></li>
<li><code>/tip</code> , <code>/blocks</code>, <code>/block_info</code> =&gt; <code>block_time</code></li>
<li><code>/genesis</code> =&gt; <code>systemStart</code></li>
<li><code>/epoch_info</code> =&gt; <code>start_time</code>, <code>first_block_time</code>, <code>last_block_time</code>, <code>end_time</code></li>
<li><code>/tx_info</code> =&gt; <code>tx_timestamp</code></li>
<li><code>/asset_info</code> =&gt; <code>creation_time</code></li>
<li><code>/blocks</code>, <code>/block_info</code> =&gt; Add <code>proto_major</code> and <code>proto_minor</code> for a given block to output <a href="https://github.com/cardano-community/koios-artifacts/pull/55">#55</a></li>
</ul>
<h3 id="changes-for-instance-providers_1">Changes for Instance Providers<a class="headerlink" href="#changes-for-instance-providers_1" title="Permanent link">⚓︎</a></h3>
<ul>
<li>For consistency between date formats, we highly recommend you to upgrade your instance to use Postgres 14 (prolly a good time, since you would already need to recreate DB for dbsync v13). You can find sample instructions to do so <a href="https://www.paulox.net/2022/04/28/upgrading-postgresql-from-version-13-to-14-on-ubuntu-22-04-jammy-jellyfish/">here</a></li>
<li>Various changes to backend scripts and performance optimisations that can be found <a href="https://github.com/cardano-community/guild-operators/compare/koios-1.0.1...koios-1.0.5">here</a></li>
</ul>
<h2 id="101">[1.0.1]<a class="headerlink" href="#101" title="Permanent link">⚓︎</a></h2>
<ul>
<li>Modify <code>asset_registry_update.sh</code> script to rely on commit hash instead of POSIX timestamps, and performance bump. <a href="https://github.com/cardano-community/guild-operators/pull/1428">#1428</a></li>
Expand All @@ -1065,8 +1280,8 @@ <h2 id="100">[1.0.0]<a class="headerlink" href="#100" title="Permanent link">⚓
<li>First Production release for Koios gRest</li>
</ul>
<h2 id="100-rc1">[1.0.0-rc1]<a class="headerlink" href="#100-rc1" title="Permanent link">⚓︎</a></h2>
<h3 id="changes-for-api">Changes for API<a class="headerlink" href="#changes-for-api" title="Permanent link">⚓︎</a></h3>
<h4 id="data-output-changes">Data Output Changes<a class="headerlink" href="#data-output-changes" title="Permanent link">⚓︎</a></h4>
<h3 id="changes-for-api_2">Changes for API<a class="headerlink" href="#changes-for-api_2" title="Permanent link">⚓︎</a></h3>
<h4 id="data-output-changes_2">Data Output Changes<a class="headerlink" href="#data-output-changes_2" title="Permanent link">⚓︎</a></h4>
<ul>
<li>Improve: Add <code>epoch_no</code>, <code>block_no</code> to <code>/address_txs</code>, <code>/credential_txs</code> and <code>/asset_txs</code> endpoints. <a href="https://github.com/cardano-community/guild-operators/pull/1403">#1409</a></li>
<li>Fix: Remove redundant policy_info for <code>/asset_txs</code>, returning transactions as an array - allows for leveraging native PostgREST filtering. <a href="https://github.com/cardano-community/guild-operators/pull/1403">#1409</a></li>
Expand All @@ -1076,7 +1291,7 @@ <h4 id="input-parameter-changes">Input Parameter Changes<a class="headerlink" hr
<ul>
<li>None</li>
</ul>
<h3 id="changes-for-instance-providers">Changes for Instance Providers<a class="headerlink" href="#changes-for-instance-providers" title="Permanent link">⚓︎</a></h3>
<h3 id="changes-for-instance-providers_2">Changes for Instance Providers<a class="headerlink" href="#changes-for-instance-providers_2" title="Permanent link">⚓︎</a></h3>
<h4 id="added">Added<a class="headerlink" href="#added" title="Permanent link">⚓︎</a></h4>
<ul>
<li>Add setup-grest.sh versioning. When running setup-grest.sh against a branch/tag, it will now populate the version information on control table, the health checks will be able to use this versioning for downstream connections. <a href="https://github.com/cardano-community/guild-operators/pull/1403">#1403</a></li>
Expand Down

0 comments on commit 610fb28

Please sign in to comment.