Skip to content
This repository has been archived by the owner on Sep 30, 2021. It is now read-only.

Commit

Permalink
Update repos to point to 'mozilla' org
Browse files Browse the repository at this point in the history
  • Loading branch information
ahal committed Oct 8, 2020
1 parent e031e8e commit d5a2491
Show file tree
Hide file tree
Showing 8 changed files with 32 additions and 32 deletions.
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -69,7 +69,7 @@ you should expect something like

There is no PyPi install. Please clone `master` branch off of Github:

git clone https://github.com/klahnakoski/ActiveData.git
git clone https://github.com/mozilla/ActiveData.git
git checkout master

and install your requirements:
Expand Down
2 changes: 1 addition & 1 deletion active_data/public/index.html
Expand Up @@ -9,7 +9,7 @@
<h1>You Win!</h1>
You have discovered an instance of the ActiveData service!<br>
<a href="https://wiki.mozilla.org/Auto-tools/Projects/ActiveData">Read more on the Wiki</a><br>
<a href="https://github.com/klahnakoski/ActiveData">Look at the code</a><br>
<a href="https://github.com/mozilla/ActiveData">Look at the code</a><br>
<a href="/tools/query.html">Use the ActiveData&nbsp;Query&nbsp;Tool<br></a>
</div>
</body>
Expand Down
18 changes: 9 additions & 9 deletions active_data/public/query.html
Expand Up @@ -29,19 +29,19 @@
<div id="description">
Sends given query to <code style="font-size: 0.9em">/query</code> endpoint, and shows response.
<ul>
<li><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/GettingStarted.md">Getting Started</a></li><br>
<li><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_tutorial.md">Unittest Tutorial</a></li>
<li><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx.md">Main Documentation</a></li>
<li><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_reference.md">Reference Documentation</a></li>
<li><a href="https://github.com/klahnakoski/ActiveData">Code on Github</a></li>
<li><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/GettingStarted.md">Getting Started</a></li><br>
<li><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/jx_tutorial.md">Unittest Tutorial</a></li>
<li><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/jx.md">Main Documentation</a></li>
<li><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/jx_reference.md">Reference Documentation</a></li>
<li><a href="https://github.com/mozilla/ActiveData">Code on Github</a></li>
</ul>
<hr>
<b>Schemas</b>
<ul>
<li class="code"><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/Unittest%20Schema.md">unittest</a></li>
<li class="code"><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/Jobs%20Schema.md">jobs</a></li>
<li class="code"><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/Perf%20Schema.md">perf</a></li>
<li class="code"><a href="https://github.com/klahnakoski/ActiveData/blob/dev/docs/TaskCluster%20Schema.md">taskcluster</a></li>
<li class="code"><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/Unittest%20Schema.md">unittest</a></li>
<li class="code"><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/Jobs%20Schema.md">jobs</a></li>
<li class="code"><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/Perf%20Schema.md">perf</a></li>
<li class="code"><a href="https://github.com/mozilla/ActiveData/blob/dev/docs/TaskCluster%20Schema.md">taskcluster</a></li>
</ul>
</div>
</div>
Expand Down
8 changes: 4 additions & 4 deletions contribute.json
Expand Up @@ -2,15 +2,15 @@
"name": "ActiveData",
"description": "Billions of Records! Available to query!",
"repository": {
"url": "https://github.com/klahnakoski/ActiveData",
"url": "https://github.com/mozilla/ActiveData",
"license": "MPL2",
"type": "git",
"tests": "https://github.com/klahnakoski/ActiveData/tree/dev/tests",
"clone": "https://github.com/klahnakoski/ActiveData.git"
"tests": "https://github.com/mozilla/ActiveData/tree/dev/tests",
"clone": "https://github.com/mozilla/ActiveData.git"
},
"participate": {
"home": "https://wiki.mozilla.org/EngineeringProductivity/Projects/ActiveData",
"docs": "https://github.com/klahnakoski/ActiveData/tree/dev/docs"
"docs": "https://github.com/mozilla/ActiveData/tree/dev/docs"
},
"urls": {
"stage": "https://activedata.allizom.org"
Expand Down
12 changes: 6 additions & 6 deletions docs/GettingStarted.md
Expand Up @@ -57,9 +57,9 @@ Knowing the column names and types is not enough. Knowing the set-of-values tha

<div style="text-align:right;"><a href="http://activedata.allizom.org/tools/query.html#query_id=IkrCzx5d">http://activedata.allizom.org/tools/query.html#query_id=IkrCzx5d</a></div>

ActiveData accepts [JSON Query Expressions](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx.md), which are JSON objects with multiple properties, called "clauses" in tribute to the SQL language it tries to mimic. The above query is the simplest query you can send ActiveData: It has one clause, the `from` clause, and it returns (limited to 10) JSON documents found in the `jobs` table. In this case, ActiveData returns JSON documents, but we will always call them "records" to stay consistent with database terminology.
ActiveData accepts [JSON Query Expressions](https://github.com/mozilla/ActiveData/blob/dev/docs/jx.md), which are JSON objects with multiple properties, called "clauses" in tribute to the SQL language it tries to mimic. The above query is the simplest query you can send ActiveData: It has one clause, the `from` clause, and it returns (limited to 10) JSON documents found in the `jobs` table. In this case, ActiveData returns JSON documents, but we will always call them "records" to stay consistent with database terminology.

The records returned from ActiveData are usually quite large, so you will need a large-screen JSON formatting tool to view the query result. There is a page that has [details about what is in a `jobs` record](https://github.com/klahnakoski/ActiveData/blob/dev/docs/Jobs%20Schema.md), but we will focus on just a few columns for now.
The records returned from ActiveData are usually quite large, so you will need a large-screen JSON formatting tool to view the query result. There is a page that has [details about what is in a `jobs` record](https://github.com/mozilla/ActiveData/blob/dev/docs/Jobs%20Schema.md), but we will focus on just a few columns for now.

Here is an example result, with the `data` property collapsed:

Expand Down Expand Up @@ -98,7 +98,7 @@ There are four major properties:
* **data** - the result of the query, in the format specified by the `meta.format` property.

> ActiveData's columns are not limited to primitive types; JSON objects are treated as
> values too. You can read more about [how to use the `select` clause to shape data](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_clause_select.md),
> values too. You can read more about [how to use the `select` clause to shape data](https://github.com/mozilla/ActiveData/blob/dev/docs/jx_clause_select.md),
> but it is not necessary for the purpose of this guide.
Expanding the `data` property a couple of levels, we can see:
Expand Down Expand Up @@ -279,9 +279,9 @@ Let us restrict ourselves to `mozilla-inbound`, the principal branch for Firefox

Individual records are useful for understanding the data and building a query. They are also useful to the people that are familiar with the events that produced those records. But, you will need aggregate statistics to get a better holistic understanding of trends.

Once you are confident your `where` clause is focused on the records you're interested in, you can start requesting aggregates. Up to this point in this guide, ActiveData has either returned individual records, or provided a `count` (which is the default aggregate). [We can request many other aggregates](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_reference.md#selectaggregate-subclause)
Once you are confident your `where` clause is focused on the records you're interested in, you can start requesting aggregates. Up to this point in this guide, ActiveData has either returned individual records, or provided a `count` (which is the default aggregate). [We can request many other aggregates](https://github.com/mozilla/ActiveData/blob/dev/docs/jx_reference.md#selectaggregate-subclause)

In the following query we are going to use `edges`, which acts much like `groupby`, [see more](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_clause_edges.md), but accepts an explicit `domain` to join-and-group the data. In this case, we are bucketing the past month of `action.start_time` to days. We also define a `select` clause to give us the `average` duration, and the `count` for each of those buckets.
In the following query we are going to use `edges`, which acts much like `groupby`, [see more](https://github.com/mozilla/ActiveData/blob/dev/docs/jx_clause_edges.md), but accepts an explicit `domain` to join-and-group the data. In this case, we are bucketing the past month of `action.start_time` to days. We also define a `select` clause to give us the `average` duration, and the `count` for each of those buckets.

{
"from":"jobs",
Expand Down Expand Up @@ -341,4 +341,4 @@ Similarly for other languages and tools.

## Summary

I hope I have given you a good sense for how to explore the ActiveData schema, and how to build a query that will give you the data you are interested in. More specific documentation regarding the intricacies of JSON Query Expressions, and ActiveData, is all centrally linked on the [main documentation page](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx.md)
I hope I have given you a good sense for how to explore the ActiveData schema, and how to build a query that will give you the data you are interested in. More specific documentation regarding the intricacies of JSON Query Expressions, and ActiveData, is all centrally linked on the [main documentation page](https://github.com/mozilla/ActiveData/blob/dev/docs/jx.md)
10 changes: 5 additions & 5 deletions docs/Outreachy Proposal 170223.md
Expand Up @@ -5,8 +5,8 @@

ActiveData is a publicly accessible data warehouse holding many billions of records, for some dozen+ datasets concerning Mozilla's testing infrastructure: This includes test results, job results, code coverage, and extracts from other systems. The ActiveData code itself is really only a stateless query translation layer; leaving the hard work of high speed filtering and aggregation to Elasticsearch.

* ActiveData accepts [JSON Query Expressions](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx.md)
* [ActiveData Tutorial](https://github.com/klahnakoski/ActiveData/blob/dev/docs/GettingStarted.md) will help you understand how it is used.
* ActiveData accepts [JSON Query Expressions](https://github.com/mozilla/ActiveData/blob/dev/docs/jx.md)
* [ActiveData Tutorial](https://github.com/mozilla/ActiveData/blob/dev/docs/GettingStarted.md) will help you understand how it is used.


## Background
Expand Down Expand Up @@ -62,7 +62,7 @@ Some particular experience will make this task easier (most important first):
**What is the first step?**

>Clone the [master branch of ActiveData](https://github.com/klahnakoski/ActiveData/tree/master), follow the directions and ensure the tests pass. The tests take about 9 minutes on my machine. Feel free to ask questions if something goes wrong.
>Clone the [master branch of ActiveData](https://github.com/mozilla/ActiveData/tree/master), follow the directions and ensure the tests pass. The tests take about 9 minutes on my machine. Feel free to ask questions if something goes wrong.
>
>Once the tests pass. Upgrade your local Elasticsearch from 1.7.x to version 5.x. Run the tests again to see all the failures.
>
Expand All @@ -82,8 +82,8 @@ Some particular experience will make this task easier (most important first):
>
><pre> {"a": 1}<br> {"a": [{"b":1}, {"b":2}]}</pre>
>
> How can we make them fit into one schema? Can we somehow re-write the JSON to a new format so they will fit into ES. How do we change the Elasticsearch queries to work with the new format? [I have added a test that ensure these schemas can be merged](https://github.com/klahnakoski/ActiveData/blob/dev/tests/test_jx/test_schema_merging.py#L22)
> How can we make them fit into one schema? Can we somehow re-write the JSON to a new format so they will fit into ES. How do we change the Elasticsearch queries to work with the new format? [I have added a test that ensure these schemas can be merged](https://github.com/mozilla/ActiveData/blob/dev/tests/test_jx/test_schema_merging.py#L22)
**What branch do I work off of?**

> Work with the `master` branch; it is working and stable. The `dev` branch (which you are reading right now) is unstable and has a long way to go before merging back into master.
> Work with the `master` branch; it is working and stable. The `dev` branch (which you are reading right now) is unstable and has a long way to go before merging back into master.
10 changes: 5 additions & 5 deletions docs/_Caring_for_ActiveData.md
Expand Up @@ -24,7 +24,7 @@ Once you are familiar with this doc, then you will only need the reference doc:
Tending to ActiveData will require

* **ActiveData code** - `git clone https://github.com/mozilla/ActiveData.git`
* **ActiveData-ETL code** - `git clone https://github.com/klahnakoski/ActiveData-ETL.git`
* **ActiveData-ETL code** - `git clone https://github.com/mozilla/ActiveData-ETL.git`
* **Elasticsearch Head** - `git clone https://github.com/mobz/elasticsearch-head.git`
* **esShardBalancer** - `git clone https://github.com/klahnakoski/esShardBalancer.git`

Expand Down Expand Up @@ -208,8 +208,8 @@ Updating the web server is relatively easy

Overall, the **Manager** machine is responsible for running CRON jobs against ActiveData. The code for these jobs are found on multiple repositories, under the `manager` branch.

* The [Manager setup](https://github.com/klahnakoski/ActiveData-ETL/blob/manager/resources/scripts/setup_manager.sh) reveals the repositories being used
* [CRON jobs](https://github.com/klahnakoski/ActiveData-ETL/blob/manager/resources/cron/manager.cron) is the list of actions being performed
* The [Manager setup](https://github.com/mozilla/ActiveData-ETL/blob/manager/resources/scripts/setup_manager.sh) reveals the repositories being used
* [CRON jobs](https://github.com/mozilla/ActiveData-ETL/blob/manager/resources/cron/manager.cron) is the list of actions being performed
* Logs are found at `/data1/logs`

#### Shard Balancer
Expand All @@ -234,8 +234,8 @@ This is a simple machine running a simple program the listens to the taskcluster

The ETL is covered by two projects

* [ActiveData-ETL](https://github.com/klahnakoski/ActiveData-ETL) (using the `etl` branch) - is the main program
* [SpotManager](https://github.com/klahnakoski/SpotManager) (using the `manager` branch) - responsible for deploying instances of the above
* [ActiveData-ETL](https://github.com/mozilla/ActiveData-ETL) (using the `etl` branch) - is the main program
* [SpotManager](https://github.com/mozilla/ActiveData-SpotManager) (using the `manager` branch) - responsible for deploying instances of the above

#### Upgrading existing pipelines

Expand Down
2 changes: 1 addition & 1 deletion docs/compare_GraphQL.md
Expand Up @@ -268,7 +268,7 @@ with variables...
"withFriends": false
}

JSON Queries do not have directives. Instead, use a standard expression, albeit with constants. The [`when` expression](https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_expressions.md#when-operator) can provide identical switching logic used by the `@include(if: .)` idiom:
JSON Queries do not have directives. Instead, use a standard expression, albeit with constants. The [`when` expression](https://github.com/mozilla/ActiveData/blob/dev/docs/jx_expressions.md#when-operator) can provide identical switching logic used by the `@include(if: .)` idiom:

vars = {
"episode": "JEDI",
Expand Down

0 comments on commit d5a2491

Please sign in to comment.