Fmean, mean overload functions implemented #877

Madhav2310 · 2022-08-03T04:07:40Z

No description provided.

integration_tests/test_statistics.py

certik · 2022-08-03T06:45:31Z

Currently this test is not enabled in llvm:

RUN(NAME test_statistics     LABELS cpython)

Can you isolate all bugs from this that LPython still does not support and report them? Let's get them fixed.

I would recommend to only implement things that actually fully compile with LPython. Otherwise we'll end up with a huge library that does not work.

Smit-create · 2022-08-03T08:50:55Z

src/runtime/statistics.py

+    for i in range(k):
+        for j in range(i+1,k):
+            if x[i]>x[j]:
+                x[i],x[j]=x[j],x[i]


This code is repeated many times. How about creating a new function, say _sort and reuse it?

I made the following function

def _sort(x: list[i32]): k: i32 = len(x) i: i32 j: i32 for i in range(k): for j in range(i+1,k): if x[i]>x[j]: x[i],x[j]=x[j],x[i]

and passed the list by reference in the following manner:

_sort({x})

However, when I try to overload the sort function for f64, it throws an error. Any suggestions for alternative workarounds? @Smit-create

czgdp1807 · 2022-08-03T09:31:19Z

Lists are supported now in LLVM. So please enable LLVM for your test.

Madhav2310 · 2022-08-04T21:46:15Z

Lists are supported now in LLVM. So please enable LLVM for your test.

I'm getting the following error when enabling LLVM for this:

[ 85%] Generating test_statistics.o
Internal Compiler Error: Unhandled exception
Traceback (most recent call last):
  Binary file "/home/madhav/lpython/src/bin/lpython", in _start()
  Binary file "/lib/x86_64-linux-gnu/libc.so.6", in __libc_start_main()
  File "/home/madhav/lpython/src/bin/lpython.cpp", line 867, in ??
    return compile_python_to_object_file(arg_file, outfile, runtime_library_dir, lpython_pass_manager, compiler_options, time_report);
  File "/home/madhav/lpython/src/bin/lpython.cpp", line 350, in ??
    compiler_options.disable_main, compiler_options.symtab_only, infile);
  File "/home/madhav/lpython/src/lpython/semantics/python_ast_to_asr.cpp", line 4166, in LFortran::LPython::python_ast_to_asr(Allocator&, LFortran::LPython::AST::ast_t&, LFortran::diag::Diagnostics&, bool, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
    LFORTRAN_ASSERT(asr_verify(*tu));
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 541, in LFortran::asr_verify(LFortran::ASR::TranslationUnit_t const&, bool)
    v.visit_TranslationUnit(unit);
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 111, in LFortran::ASR::VerifyVisitor::visit_TranslationUnit(LFortran::ASR::TranslationUnit_t const&)
    this->visit_symbol(*a.second);
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3694, in LFortran::ASR::BaseVisitor<LFortran::ASR::VerifyVisitor>::visit_symbol(LFortran::ASR::symbol_t const&)
    void visit_symbol(const symbol_t &b) { visit_symbol_t(b, self()); }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3480, in ??
    case symbolType::Module: { v.visit_Module((const Module_t &)x); return; }
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 217, in LFortran::ASR::VerifyVisitor::visit_Module(LFortran::ASR::Module_t const&)
    this->visit_symbol(*a.second);
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3694, in LFortran::ASR::BaseVisitor<LFortran::ASR::VerifyVisitor>::visit_symbol(LFortran::ASR::symbol_t const&)
    void visit_symbol(const symbol_t &b) { visit_symbol_t(b, self()); }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3482, in ??
    case symbolType::Function: { v.visit_Function((const Function_t &)x); return; }
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 280, in LFortran::ASR::VerifyVisitor::visit_Function(LFortran::ASR::Function_t const&)
    visit_stmt(*x.m_body[i]);
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3708, in LFortran::ASR::BaseVisitor<LFortran::ASR::VerifyVisitor>::visit_stmt(LFortran::ASR::stmt_t const&)
    void visit_stmt(const stmt_t &b) { visit_stmt_t(b, self()); }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3513, in ??
    case stmtType::If: { v.visit_If((const If_t &)x); return; }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 4006, in LFortran::ASR::BaseWalkVisitor<LFortran::ASR::VerifyVisitor>::visit_If(LFortran::ASR::If_t const&)
    self().visit_expr(*x.m_test);
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3750, in LFortran::ASR::BaseVisitor<LFortran::ASR::VerifyVisitor>::visit_expr(LFortran::ASR::expr_t const&)
    void visit_expr(const expr_t &b) { visit_expr_t(b, self()); }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3557, in ??
    case exprType::IntegerCompare: { v.visit_IntegerCompare((const IntegerCompare_t &)x); return; }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 4317, in LFortran::ASR::BaseWalkVisitor<LFortran::ASR::VerifyVisitor>::visit_IntegerCompare(LFortran::ASR::IntegerCompare_t const&)
    self().visit_expr(*x.m_left);
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3750, in LFortran::ASR::BaseVisitor<LFortran::ASR::VerifyVisitor>::visit_expr(LFortran::ASR::expr_t const&)
    void visit_expr(const expr_t &b) { visit_expr_t(b, self()); }
  File "/home/madhav/lpython/src/libasr/../libasr/asr.h", line 3550, in ??
    case exprType::FunctionCall: { v.visit_FunctionCall((const FunctionCall_t &)x); return; }
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 498, in LFortran::ASR::VerifyVisitor::visit_FunctionCall(LFortran::ASR::FunctionCall_t const&)
    require(symtab_in_scope(current_symtab, x.m_name),
  File "/home/madhav/lpython/src/libasr/asr_verify.cpp", line 60, in LFortran::ASR::VerifyVisitor::require(bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, LFortran::Location const&)
    require(cond, msg);
LCompilersException: ASR verify failed: 2822:2824: FunctionCall::m_name `_mod@__lpython_overloaded_0___mod` cannot point outside of its symbol table
make[2]: *** [CMakeFiles/test_statistics.dir/build.make:74: test_statistics.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2012: CMakeFiles/test_statistics.dir/all] Error 2
make: *** [Makefile:101: all] Error 2

I tried to understand why the symbol table it throwing an error but couldn't figure it out. Suggestions please? @czgdp1807

czgdp1807 · 2022-08-05T08:57:20Z

The error is not related lists. It is due to incorrect symbol being used in the FunctionCall to mod. This needs some investigation. @Smit-create Can you please try looking into it?

Smit-create · 2022-08-05T09:25:42Z

Yeah, will look into it.

Smit-create · 2022-08-08T16:25:42Z

I have removed the median functions because the diff was getting larger and had some issues to fix. Rest all is working fine.

certik

That's excellent that such list usage already works.

Thanks!

…mented

czgdp1807 · 2022-08-09T10:41:42Z

Great. Thanks @Smit-create for working on this and completing this.

Madhav2310 · 2022-08-09T10:50:56Z

I have removed the median functions because the diff was getting larger and had some issues to fix. Rest all is working fine.

@Smit-create could you please explain what you mean by this? That way I'll be careful of the issue in question as I implement further functions

Madhav2310 · 2022-08-09T10:52:06Z

I have removed the median functions because the diff was getting larger and had some issues to fix. Rest all is working fine.

@Smit-create could you please explain what you mean by this? That way I'll be careful of the issue in question as I implement further functions

How could I improve on the median function? I also have another PR open with Variance and stdev functions.

certik · 2022-08-09T13:04:09Z

src/runtime/statistics.py

+
+    for i in range(k):
+        sum += x[i]
+    return sum/k


Btw, these are great candidates to try using the new templates. The templates are still experimental, and we still need to implement restrictions, so things will change, but the current state should be able to compile such functions.

I agree. @ansharlubis @Oshanath May be, you can try templates on these functions and reduce the implementation size? What do you folks say?

I think it might be a little bit early, since it will slow @ansharlubis down if we have to do any refactoring, which we will. We can however do it on a branch / Draft PR that is not merged. That would be great, just to see if it works, and to iron out any bugs, but I would not merge into master yet.

It kind of work in an example I added in PR #921, it still uses the hard-coded restrictions though and don't do much type checks.

namannimmo10 reviewed Aug 3, 2022

View reviewed changes

integration_tests/test_statistics.py Show resolved Hide resolved

Smit-create reviewed Aug 3, 2022

View reviewed changes

Smit-create force-pushed the Statistics2 branch from 909f616 to 065e271 Compare August 8, 2022 16:24

Smit-create changed the title ~~Fmean, Median, Median_low, median_high, mean overload functions implemented~~ Fmean, mean overload functions implemented Aug 8, 2022

Smit-create requested a review from czgdp1807 August 8, 2022 16:26

certik approved these changes Aug 8, 2022

View reviewed changes

Madhav2310 and others added 5 commits August 9, 2022 15:57

fmean, Median, Median_low, median_high, mean overload functions imple…

928408b

…mented

Implement list argument in LLVM

a914135

Fix list overload error

d535b29

Add support for mean, fmean, geometric/harmonic mean

88201d5

Add and update tests

a3af2ab

Smit-create force-pushed the Statistics2 branch from 065e271 to a3af2ab Compare August 9, 2022 10:31

Smit-create enabled auto-merge August 9, 2022 10:31

Smit-create merged commit 4a4d406 into lcompilers:main Aug 9, 2022

certik reviewed Aug 9, 2022

View reviewed changes

Madhav2310 mentioned this pull request Aug 12, 2022

Initial implementation of statistics.mode() #958

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fmean, mean overload functions implemented #877

Fmean, mean overload functions implemented #877

Madhav2310 commented Aug 3, 2022 •

edited

certik commented Aug 3, 2022

Smit-create Aug 3, 2022

Madhav2310 Aug 4, 2022 •

edited

czgdp1807 commented Aug 3, 2022

Madhav2310 commented Aug 4, 2022

czgdp1807 commented Aug 5, 2022

Smit-create commented Aug 5, 2022

Smit-create commented Aug 8, 2022

certik left a comment

czgdp1807 commented Aug 9, 2022

Madhav2310 commented Aug 9, 2022

Madhav2310 commented Aug 9, 2022

certik Aug 9, 2022

czgdp1807 Aug 12, 2022

certik Aug 12, 2022

ansharlubis Aug 15, 2022

Fmean, mean overload functions implemented #877

Fmean, mean overload functions implemented #877

Conversation

Madhav2310 commented Aug 3, 2022 • edited

certik commented Aug 3, 2022

Smit-create Aug 3, 2022

Choose a reason for hiding this comment

Madhav2310 Aug 4, 2022 • edited

Choose a reason for hiding this comment

czgdp1807 commented Aug 3, 2022

Madhav2310 commented Aug 4, 2022

czgdp1807 commented Aug 5, 2022

Smit-create commented Aug 5, 2022

Smit-create commented Aug 8, 2022

certik left a comment

Choose a reason for hiding this comment

czgdp1807 commented Aug 9, 2022

Madhav2310 commented Aug 9, 2022

Madhav2310 commented Aug 9, 2022

certik Aug 9, 2022

Choose a reason for hiding this comment

czgdp1807 Aug 12, 2022

Choose a reason for hiding this comment

certik Aug 12, 2022

Choose a reason for hiding this comment

ansharlubis Aug 15, 2022

Choose a reason for hiding this comment

Madhav2310 commented Aug 3, 2022 •

edited

Madhav2310 Aug 4, 2022 •

edited