Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement f-string support #3250

Open
marcino239 opened this issue Aug 22, 2018 · 22 comments
Open

Implement f-string support #3250

marcino239 opened this issue Aug 22, 2018 · 22 comments

Comments

@marcino239
Copy link

marcino239 commented Aug 22, 2018

This program:

import numba

@numba.jit(nopython=False)
def test( a: int ):
    b = a + 1
    print( f'b:{b}' )
    return b

b = test( 1 )
print( f'b:{b}' )

Throws this exception:

(py36) [marcino@localhost numbatest]$ python test.py 
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    b = test( 1 )
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dispatcher.py", line 368, in _compile_for_args
    raise e
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dispatcher.py", line 325, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dispatcher.py", line 653, in compile
    cres = self._compiler.compile(args, return_type)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dispatcher.py", line 83, in compile
    pipeline_class=self.pipeline_class)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 873, in compile_extra
    return pipeline.compile_extra(func)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 367, in compile_extra
    return self._compile_bytecode()
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 804, in _compile_bytecode
    return self._compile_core()
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 791, in _compile_core
    res = pm.run(self.status)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 253, in run
    raise patched_exception
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 245, in run
    stage()
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 381, in stage_analyze_bytecode
    func_ir = translate_stage(self.func_id, self.bc)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/compiler.py", line 937, in translate_stage
    return interp.interpret(bytecode)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/interpreter.py", line 98, in interpret
    self.dfa.run()
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dataflow.py", line 28, in run
    self.infos[blk.offset] = self.run_on_block(blk)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dataflow.py", line 78, in run_on_block
    self.dispatch(info, inst)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dataflow.py", line 88, in dispatch
    fn(info, inst)
  File "/home/marcino/py36/lib/python3.6/site-packages/numba/dataflow.py", line 93, in handle_unknown_opcode
    self.bytecode.func_id.filename))
NotImplementedError: Failed at object (analyzing bytecode)
Use of unknown opcode FORMAT_VALUE at line 6 of test.py
@marcino239 marcino239 changed the title Numba, use of unknown opcode FORMAT_VALUE, python 3.6 Numba 0.39.0, use of unknown opcode FORMAT_VALUE, python 3.6 Aug 22, 2018
@stuartarchibald
Copy link
Contributor

Thanks for the report. The FORMAT_VALUE op code isn't supported (as noted by the error message), adding this as a feature request.

@rtmlp
Copy link

rtmlp commented Aug 27, 2018

If you don't mid asking, what is the error message trying to mention. I got similar sort of error but don't know how to understand the error message

@anthonytec2
Copy link
Contributor

anthonytec2 commented Aug 31, 2018

For implementing the FORMAT_VALUE opcode, would this just require adding an opcode to the interpreter.py file? What further is required to fix this up?

@stuartarchibald
Copy link
Contributor

@rtmatx the error message is mentioning that the opcode FORMAT_VALUE is not supported by Numba. The use of opcode refers to the "operation code" for a given bytecode instruction https://docs.python.org/3/library/dis.html#python-bytecode-instructions. Numba works by first interpreting the Python bytecode through the use of opcodes, more can be read about this here http://numba.pydata.org/numba-doc/dev/developer/architecture.html?highlight=bytecode#stage-1-analyze-bytecode.

@stuartarchibald
Copy link
Contributor

@anthonytec2 The FORMAT_VALUE opcode would need adding to the interpreter+dataflow, as would BUILD_STRING. If this was to work under nopython mode then a load of typing and implementation lowering would also need writing, I imagine that this would be quite challenging.

@esc
Copy link
Member

esc commented Apr 3, 2019

#3938 implements a better error message in such cases. For example, in the case above it will print:

💥 zsh» python v3.py                                                                                                                                          :(
Traceback (most recent call last):
  File "v3.py", line 9, in <module>
    b = test( 1 )
  File "/Users/vhaenel/git/numba/numba/dispatcher.py", line 353, in _compile_for_args
    error_rewrite(e, 'unsupported_error')
  File "/Users/vhaenel/git/numba/numba/dispatcher.py", line 317, in error_rewrite
    reraise(type(e), e, None)
  File "/Users/vhaenel/git/numba/numba/six.py", line 658, in reraise
    raise value.with_traceback(tb)
numba.errors.UnsupportedError: Failed in object mode pipeline (step: analyzing bytecode)
Use of unknown opcode 'FORMAT_VALUE'

File "v3.py", line 6:
def test( a: int ):
    <source elided>
    b = a + 1
    print( f'b:{b}' )
    ^

Unsupported functionality was found in the code Numba was trying to compile.

If this functionality is important to you please file a feature request at:
https://github.com/numba/numba/issues/new

@zillionare
Copy link

hi guys, any workaround?

@stuartarchibald
Copy link
Contributor

@hbaaron unfortunately there is no work around, use of format string (and hence that opcode) is not yet supported.

@monocongo
Copy link

I have been using formatted strings (f"string like this with {values}") and everything has worked well until I saw this error today when running some code that was running well before. Maybe the previous usages of my numba annotated functions were not exercising the blocks where I have the formatted strings that cause this error? My point is that I have formatted strings throughout my code, most of which is annotated with @numba.jit, so I was surprised by this error being raised in code when I hadn't modified it recently. Must I remove all formatted strings in order to be safe from this issue? It seems that changing the offending lines to instead use format() does indeed fix things, but it's an unwelcome workaround for sure.

monocongo added a commit to monocongo/climate_indices that referenced this issue Apr 30, 2019
…dError from numba

NotImplementedError: Failed in object mode pipeline (step: analyzing bytecode)
Use of unknown opcode FORMAT_VALUE

see numba/numba#3250

#306
@stuartarchibald
Copy link
Contributor

@monocongo I'm not sure what's going on in your case, @numba.jit would have to compile the blocks irrespective of whether they are used, which would require it interpreting bytecode containing the FORMAT_VALUE op code. e.g.:

In [1]: def foo(values): 
   ...:     return f"string {values}" 
   ...:                                                                                                                         

In [2]: def bar(values): 
   ...:     return "string {}".format(values) 
   ...:                                                                                                                         

In [3]: from dis import dis                                                                                                     

In [4]: dis(foo)                                                                                                                
  2           0 LOAD_CONST               1 ('string ')
              2 LOAD_FAST                0 (values)
              4 FORMAT_VALUE             0
              6 BUILD_STRING             2
              8 RETURN_VALUE

In [5]: from numba import jit                                                                                                   

In [6]: jit(foo)(10)                                                                                                            
---------------------------------------------------------------------------
UnsupportedError: Failed in object mode pipeline (step: analyzing bytecode)
Use of unknown opcode 'FORMAT_VALUE'

File "<ipython-input-1-2130adc6315a>", line 2:
def foo(values):
    return f"string {values}"
    ^

Unsupported functionality was found in the code Numba was trying to compile.

If this functionality is important to you please file a feature request at:
https://github.com/numba/numba/issues/new


In [7]: jit(bar)(10)                                                                                                            
Out[7]: 'string 10'

In [8]: jit(nopython=True)(bar)(10)                                                                                             
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'format' of type Literal[str](string {})

File "<ipython-input-2-852f47965480>", line 2:
def bar(values):
    return "string {}".format(values)
    ^

[1] During: typing of get attribute at <ipython-input-2-852f47965480> (2)

File "<ipython-input-2-852f47965480>", line 2:
def bar(values):
    return "string {}".format(values)
    ^

In [9]: dis(bar)                                                                                                                
  2           0 LOAD_CONST               1 ('string {}')
              2 LOAD_METHOD              0 (format)
              4 LOAD_FAST                0 (values)
              6 CALL_METHOD              1
              8 RETURN_VALUE

Even with a dead branch, the op code would still need to be interpreted...

In [10]: def foo_branch(values, jmp): 
    ...:     if jmp > 0: 
    ...:         return f"string {values}" 
    ...:     else: 
    ...:         return "const str" 
    ...:          
    ...:                                                                                                                        

In [11]: dis(foo_branch)                                                                                                        
  2           0 LOAD_FAST                1 (jmp)
              2 LOAD_CONST               1 (0)
              4 COMPARE_OP               4 (>)
              6 POP_JUMP_IF_FALSE       18

  3           8 LOAD_CONST               2 ('string ')
             10 LOAD_FAST                0 (values)
             12 FORMAT_VALUE             0
             14 BUILD_STRING             2
             16 RETURN_VALUE

  5     >>   18 LOAD_CONST               3 ('const str')
             20 RETURN_VALUE
             22 LOAD_CONST               0 (None)
             24 RETURN_VALUE

In [12]: jit(foo_branch)(10, -1) 

UnsupportedError: Failed in object mode pipeline (step: analyzing bytecode)
Use of unknown opcode 'FORMAT_VALUE'

File "<ipython-input-10-c5ac4bf10bb6>", line 3:
def foo_branch(values, jmp):
    <source elided>
    if jmp > 0:
        return f"string {values}"
        ^

Also, @jit without specifying nopython=True is not necessarily beneficial http://numba.pydata.org/numba-doc/latest/user/5minguide.html#what-is-nopython-mode

monocongo added a commit to monocongo/climate_indices that referenced this issue Jun 2, 2019
gc56 added a commit to gc56/climate_indices that referenced this issue Aug 9, 2019
to work around the numba issue described here: numba/numba#3250
@esc
Copy link
Member

esc commented Sep 23, 2019

Since this is a feature request and it is probably still wanted, I'll change the title now to reflect what is being requested.

@esc esc changed the title Numba 0.39.0, use of unknown opcode FORMAT_VALUE, python 3.6 Implement f-string support Sep 23, 2019
@monocongo
Copy link

Thanks, @esc. It'll be nice to be able to use f-strings in numba-annotated code once this is ready to go. I appreciate the useful work you guys do on numba, thanks!

@esc
Copy link
Member

esc commented Sep 23, 2019

@monocongo thanks for the kind words! We appreciate you using Numba. 😄

@adizhol
Copy link

adizhol commented Dec 8, 2020

still no support?

@esc
Copy link
Member

esc commented Dec 8, 2020

@adizhol that is correct, so sorry.

@mark03
Copy link

mark03 commented Jan 14, 2021

Can I use print with argument 'end' like this: print('something', end='\r')?
Or can I do it in other way?

@Hvass-Labs
Copy link

Thanks for making numba, it is incredibly cool and powerful!

I am using numba as @jit(forceobj=True) because the old fallback-mode will be deprecated, and this makes my Python code easy to read and maintain, and I just need numba to optimize some for-loops. This gives me a 10x speed-up!

I would like to use f-strings but I get the same numba exception as described above.

Would it be possible to add support for f-strings when numba is running in forced object mode?

Thanks!

@stuartarchibald
Copy link
Contributor

xref #6608, which provides basic support.

@esc esc removed their assignment Feb 16, 2022
@IamShubhamGupto
Copy link

please fix this

@lucasjinreal
Copy link

5 years later, any updates?

@erdogant
Copy link

Unfortunately, I also get this error.

@esc
Copy link
Member

esc commented Dec 20, 2023

This is clearly looking for someone to volunteer to implement it. Please don't hesitate to consider this if you need it for your Numba use-case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests