Skip to content

Commit

Permalink
Merge branch 'origin/master' into release/graal-vm/1.0
Browse files Browse the repository at this point in the history
  • Loading branch information
ansalond committed Aug 28, 2018
2 parents 0590b8b + d06ed1d commit b1cb6d3
Show file tree
Hide file tree
Showing 1,254 changed files with 62,357 additions and 27,021 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ language
Python3.g4.stamp
*.orig
/graalpython/com.oracle.graal.python.test/src/tests/cpyext/Py*.c
/graalpython/com.oracle.graal.python.test/src/tests/cpyext/Test*.c
/*.diff
## generated from: pyhocon -i ci.hocon -f json -o ci.json
/ci.json
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,17 @@
This changelog summarizes major changes between GraalVM versions of the Python
language runtime. The main focus is on user-observable behavior of the engine.

## Version 1.0.0 RC5

* Generator expressions now properly evaluate their first iterator in the definition scope at definition time
* Fixes for embedders to ensure top scopes are stable and local scopes always contain TruffleObjects
* C-API improvements to support simple Cython modules
* Support recognition of Python source files with the polyglot launcher

## Version 1.0.0 RC4

* No changes

## Version 1.0.0 RC3

* Support for more String encodings
Expand Down
44 changes: 44 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
**Thank you for thinking about contributing something to this Python
implementation on Graal!**

## First things first

You will need to sign the [Oracle Contributor
Agreement](http://www.graalvm.org/community/contributors/) for us to be able to
merge your work.

Please also take some time to review our [code of
conduct](http://www.graalvm.org/community/conduct/) for contributors.

## How to contribute?

If you want to make a contribution, it's best to open an issue first about the
area you would like to offer your help with. This implementation is in an early
state and it will make more sense to work on some areas than others, so before
you sink work into a pull request that we won't be able to accept, please talk
to us :)

To get started with development on Python, first make sure you can build at
least the _truffle_ project in the [Graal](https://github.com/oracle/graal)
repository with our build tool [mx](https://github.com/graalvm/mx), as well as
[Sulong](https://github.com/graalvm/sulong).

Once you can build those projects, clone this repository and run

$ mx build
$ mx python -c "print(42)"

If this prints "42", then everything went fine and you just built and ran
Python. Note that you don't need GraalVM for this, but then you'll only run
interpreted, which won't give very good performance (but that might be fine for
development).

When you make your changes, you can test them with `mx python`. Additionally,
there are various "gates" that we use on our CI system to check any code that
goes in. You can run all of these gates using `mx python-gate` or just some by
using `mx python-gate --tags [TAG]`. Three interesting tags to run that cover
most things are:

- `python-unittest` - Run the unittests written in Python, including those for the C extension API
- `python-graalvm` - Build a minimal GraalVM bundle with Python only and a native launcher
- `python-license` - Check that all files have the correct copyright headers applied to them
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ Universal Permissive License v 1.0 as shown at
[http://oss.oracle.com/licenses/upl](http://oss.oracle.com/licenses/upl). This
implementation is in part derived from and contains additional code from 3rd
parties, the copyrights and licensing of which is detailed in the
[LICENSE](LICENSE) file.
[LICENSE](LICENSE) and [3rd_party_licenses.txt](3rd_party_licenses.txt) files.
14 changes: 10 additions & 4 deletions ci.jsonnet
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
overlay: "03b824f947adf9a67c80c3e8a27646d43a687bfb",
overlay: "2562524065c42e18d4a5ea57e8cbf6cac6b9bdda",

// ======================================================================================================
//
Expand Down Expand Up @@ -53,6 +53,9 @@
error "unknown field: "+field+" in "+object+", valid choices are: "+std.objectFields(object)
else
object[field],

graalOption: function(name, value)
["--Ja", "@-Dgraal."+name+"="+value],
},

// ------------------------------------------------------------------------------------------------------
Expand All @@ -79,7 +82,7 @@
linuxMixin: linuxMixin,

local linuxBenchMixin = linuxMixin + {
capabilities +: ["no_frequency_scaling", "tmpfs25g", "x62"],
capabilities +: ["no_frequency_scaling", "tmpfs25g", "x52"],
},
linuxBenchMixin: linuxBenchMixin,

Expand Down Expand Up @@ -178,7 +181,7 @@

setup +: [
["mx", "sforceimport"],
["mx", "-v", "--dynamicimports", self.dynamicImports, "build"],
["mx", "--dynamicimports", self.dynamicImports, "build"],
]
},
commonBuilder: commonBuilder,
Expand All @@ -191,9 +194,12 @@
local baseGate = commonBuilder + {
tags: "tags must be defined",

// local truffleDebugFlags = utils.graalOption("TraceTruffleCompilation", "true"),
// local truffleDebugFlags = utils.graalOption("TraceTruffleCompilationDetails", "true"),
local truffleDebugFlags = [],
targets: TARGET.gate,
run +: [
["mx", "--strict-compliance", "--dynamicimports", super.dynamicImports, "--primary", "gate", "--tags", self.tags, "-B=--force-deprecation-as-warning-for-dependencies"],
["mx"] + truffleDebugFlags + ["--strict-compliance", "--dynamicimports", super.dynamicImports, "--primary", "gate", "--tags", self.tags, "-B=--force-deprecation-as-warning-for-dependencies"],
]
},

Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Copyright (c) 2018, Oracle and/or its affiliates.
# Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# The Universal Permissive License (UPL), Version 1.0
#
# Subject to the condition set forth below, permission is hereby granted to any
# person obtaining a copy of this software, associated documentation and/or data
# (collectively the "Software"), free of charge and under any and all copyright
# rights in the Software, and any and all patent rights owned or freely
# licensable by each licensor hereunder covering either (i) the unmodified
# Software as contributed to or provided by such licensor, or (ii) the Larger
# Works (as defined below), to deal in both
# person obtaining a copy of this software, associated documentation and/or
# data (collectively the "Software"), free of charge and under any and all
# copyright rights in the Software, and any and all patent rights owned or
# freely licensable by each licensor hereunder covering either (i) the
# unmodified Software as contributed to or provided by such licensor, or (ii)
# the Larger Works (as defined below), to deal in both
#
# (a) the Software, and
#
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
# one is included with the Software (each a "Larger Work" to which the
# Software is contributed by such licensors),
# one is included with the Software each a "Larger Work" to which the Software
# is contributed by such licensors),
#
# without restriction, including without limitation the rights to copy, create
# derivative works of, display, perform, and distribute the Software and make,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Copyright (c) 2018, Oracle and/or its affiliates.
# Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# The Universal Permissive License (UPL), Version 1.0
#
# Subject to the condition set forth below, permission is hereby granted to any
# person obtaining a copy of this software, associated documentation and/or data
# (collectively the "Software"), free of charge and under any and all copyright
# rights in the Software, and any and all patent rights owned or freely
# licensable by each licensor hereunder covering either (i) the unmodified
# Software as contributed to or provided by such licensor, or (ii) the Larger
# Works (as defined below), to deal in both
# person obtaining a copy of this software, associated documentation and/or
# data (collectively the "Software"), free of charge and under any and all
# copyright rights in the Software, and any and all patent rights owned or
# freely licensable by each licensor hereunder covering either (i) the
# unmodified Software as contributed to or provided by such licensor, or (ii)
# the Larger Works (as defined below), to deal in both
#
# (a) the Software, and
#
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
# one is included with the Software (each a "Larger Work" to which the
# Software is contributed by such licensors),
# one is included with the Software each a "Larger Work" to which the Software
# is contributed by such licensors),
#
# without restriction, including without limitation the rights to copy, create
# derivative works of, display, perform, and distribute the Software and make,
Expand Down
20 changes: 11 additions & 9 deletions graalpython/benchmarks/src/benchmarks/sieve.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Copyright (c) 2017, 2018, Oracle and/or its affiliates.
# Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# The Universal Permissive License (UPL), Version 1.0
#
# Subject to the condition set forth below, permission is hereby granted to any
# person obtaining a copy of this software, associated documentation and/or data
# (collectively the "Software"), free of charge and under any and all copyright
# rights in the Software, and any and all patent rights owned or freely
# licensable by each licensor hereunder covering either (i) the unmodified
# Software as contributed to or provided by such licensor, or (ii) the Larger
# Works (as defined below), to deal in both
# person obtaining a copy of this software, associated documentation and/or
# data (collectively the "Software"), free of charge and under any and all
# copyright rights in the Software, and any and all patent rights owned or
# freely licensable by each licensor hereunder covering either (i) the
# unmodified Software as contributed to or provided by such licensor, or (ii)
# the Larger Works (as defined below), to deal in both
#
# (a) the Software, and
#
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
# one is included with the Software (each a "Larger Work" to which the
# Software is contributed by such licensors),
# one is included with the Software each a "Larger Work" to which the Software
# is contributed by such licensors),
#
# without restriction, including without limitation the rights to copy, create
# derivative works of, display, perform, and distribute the Software and make,
Expand Down
159 changes: 159 additions & 0 deletions graalpython/benchmarks/src/harness.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# The Universal Permissive License (UPL), Version 1.0
#
# Subject to the condition set forth below, permission is hereby granted to any
# person obtaining a copy of this software, associated documentation and/or
# data (collectively the "Software"), free of charge and under any and all
# copyright rights in the Software, and any and all patent rights owned or
# freely licensable by each licensor hereunder covering either (i) the
# unmodified Software as contributed to or provided by such licensor, or (ii)
# the Larger Works (as defined below), to deal in both
#
# (a) the Software, and
#
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
# one is included with the Software each a "Larger Work" to which the Software
# is contributed by such licensors),
#
# without restriction, including without limitation the rights to copy, create
# derivative works of, display, perform, and distribute the Software and make,
# use, sell, offer for sale, import, export, have made, and have sold the
# Software and the Larger Work(s), and to sublicense the foregoing rights on
# either these or other terms.
#
# This license is subject to the following condition:
#
# The above copyright notice and either this complete permission notice or at a
# minimum a reference to the UPL must be included in all copies or substantial
# portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import _io
import os
import sys
from time import time

_HRULE = '-'.join(['' for i in range(80)])
ATTR_BENCHMARK = '__benchmark__'
ATTR_PROCESS_ARGS = '__process_args__'


def _as_int(value):
if isinstance(value, (list, tuple)):
value = value[0]

if not isinstance(value, int):
return int(value)
return value


class BenchRunner(object):
def __init__(self, bench_file, bench_args=None, iterations=1, warmup=0):
if bench_args is None:
bench_args = []
self.bench_module = BenchRunner.get_bench_module(bench_file)
self.bench_args = bench_args
self.iterations = _as_int(iterations)
assert isinstance(self.iterations, int)
self.warmup = _as_int(warmup)
assert isinstance(self.warmup, int)

@staticmethod
def get_bench_module(bench_file):
name = bench_file.rpartition("/")[2].partition(".")[0].replace('.py', '')
directory = bench_file.rpartition("/")[0]
pkg = []
while any(f.endswith("__init__.py") for f in os.listdir(directory)):
directory, slash, postfix = directory.rpartition("/")
pkg.insert(0, postfix)

if pkg:
sys.path.insert(0, directory)
bench_module = __import__(".".join(pkg + [name]))
for p in pkg[1:]:
bench_module = getattr(bench_module, p)
bench_module = getattr(bench_module, name)
return bench_module

else:
bench_module = type(sys)(name, bench_file)
with _io.FileIO(bench_file, "r") as f:
bench_module.__file__ = bench_file
exec(compile(f.readall(), bench_file, "exec"), bench_module.__dict__)
return bench_module

def _get_attr(self, attr_name):
if hasattr(self.bench_module, attr_name):
return getattr(self.bench_module, attr_name)

def _call_attr(self, attr_name, *args):
attr = self._get_attr(attr_name)
if attr and hasattr(attr, '__call__'):
return attr(*args)

def run(self):
print(_HRULE)
print("### %s, %s warmup iterations, %s bench iterations " % (self.bench_module.__name__, self.warmup, self.iterations))

# process the args if the processor function is defined
args = self._call_attr(ATTR_PROCESS_ARGS, *self.bench_args)
if args is None:
# default args processor considers all args as ints
args = list(map(int, self.bench_args))

print("### args = %s" % args)
print(_HRULE)

bench_func = self._get_attr(ATTR_BENCHMARK)
if bench_func and hasattr(bench_func, '__call__'):
if self.warmup:
print("### warming up for %s iterations ... " % self.warmup)
for _ in range(self.warmup):
bench_func(*args)

for iteration in range(self.iterations):
start = time()
bench_func(*args)
duration = "%.3f" % (time() - start)
print("### iteration=%s, name=%s, duration=%s" % (iteration, self.bench_module.__name__, duration))


def run_benchmark(prog, args):
warmup = 0
iterations = 1
bench_file = None
bench_args = []

i = 0
while i < len(args):
arg = args[i]
if arg == '-i':
i += 1
iterations = _as_int(args[i])
elif arg.startswith("--iterations"):
iterations = _as_int(arg.split("=")[1])
elif arg == '-w':
i += 1
warmup = _as_int(args[i])
elif arg.startswith("--warmup"):
warmup = _as_int(arg.split("=")[1])
elif bench_file is None:
bench_file = arg
else:
bench_args.append(arg)
i += 1

BenchRunner(bench_file, bench_args=bench_args, iterations=iterations, warmup=warmup).run()


if __name__ == '__main__':
run_benchmark(sys.argv[0], sys.argv[1:])
Loading

0 comments on commit b1cb6d3

Please sign in to comment.