Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to get a parseable unit string #412

Open
MartinOtter opened this issue Jan 1, 2021 · 12 comments
Open

How to get a parseable unit string #412

MartinOtter opened this issue Jan 1, 2021 · 12 comments

Comments

@MartinOtter
Copy link

I have the following problem:

using Unitful
# Variable with unit defined
v = 2.0u"m/s"    

# Extract unit as string
v_unit = string(unit(v))   # = "m s^-1"

# This string cannot be parsed
uparse(v_unit)   # gives an error

# Therefore code generation with the v_unit string does not work
code = :( @u_str($v_unit) )      # = :( u"m s^-1" )
eval(code)   # Gives an error

The reason is that string(unit(v)) returns a string that cannot be parsed. In Julia this seems to be an unusual behavior because string(something) typically returns a string representation of something that can be again parsed by Julia.

I searched extensively in Unitful documentation and the Issues but did not find a solution.

I currently use a bad hack (that most likely does not work in all situations), by replacing " " by "*":

v_unit = replace(  string(unit(v)), " " => "*")    # = "m*s^-1"
uparse(v_unit)   # fine
code = :( @u_str($v_unit) )      # = :( u"m*s^-1" )
eval(code)   # fine

Help is appreciated

@hustf
Copy link

hustf commented Jan 12, 2021

I put a lenient parser here. This uses an unregistered fork of Unitful, but should be easily adaptable. It's lenient with regards to space or no space between number and unit.

@hustf
Copy link

hustf commented Jan 12, 2021

...also lenient with regards to brackets. It's used for parsing output from proprietary applications with various formats.

julia> # When parsing text file, spaces as multipliers and brackets are allowed. Just specify the numeric type:
julia> lin = "2 [s]\t11364.56982421875 [N]\t-44553.50244140625 [N]\t-26.586366176605225 [N]\t0.0[N mm]\t0.0[N mm]\t0.0[N mm]\t1561.00350618362 [mm]\t-6072.3729133606 [mm]\t2825.15907287598 [mm]"
"2 [s]\t11364.56982421875 [N]\t-44553.50244140625 [N]\t-26.586366176605225 [N]\t0.0[N mm]\t0.0[N mm]\t0.0[N mm]\t1561.00350618362 [mm]\t-6072.3729133606 [mm]\t2825.15907287598 [mm]"

julia> time, Fx, Fy, Fz, Mx, My, Mz, px, py, pz = parse.(Quantity{Float64}, split(lin, '\t'))
10-element Array{Quantity{Float64,D,U} where U where D,1}:
                 2.0s
   11364.56982421875N
  -44553.50244140625N
 -26.586366176605225N
    0.0mm∙N
    0.0mm∙N
    0.0mm∙N
   1561.00350618362mm
   -6072.3729133606mm
   2825.15907287598mm

@singularitti
Copy link
Contributor

Same question here! Unitful often prints units that itself cannot parse:

julia> u"angstrom^3"
ų

julia> uparse("ų")
ERROR: ArgumentError: Symbol ų could not be found in unit modules Module[Unitful]
Stacktrace:
 [1] lookup_units(unitmods::Vector{Module}, sym::Symbol)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:581
 [2] lookup_units
   @ ~/.julia/packages/Unitful/1t88N/src/user.jl:593 [inlined]
 [3] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:536
 [4] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:535
 [5] top-level scope
   @ REPL[5]:1

This is not helpful when I want to serialize a quantity with units into a JSON or YAML file, then reconstruct it from that file later.

@DilumAluthge
Copy link
Contributor

Bump. Is there any update on this?

I don't necessarily need a unit string that can be parsed by Julia. But I would like a unit string that can at least be parsed by Unitful.uparse.

E.g. currently none of these work:

julia> a = Unitful.u"mg/dL/dL"
mg dL⁻²

julia> string(a)
"mg dL⁻²"

julia> Unitful.uparse(string(a))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[6]:1

julia> b = Unitful.mg / Unitful.dL / Unitful.dL
mg dL⁻²

julia> a === b
true

julia> string(b)
"mg dL⁻²"

julia> Unitful.uparse(string(b))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[10]:1

julia> repr(a)
"mg dL⁻²"

julia> repr(b)
"mg dL⁻²"

julia> Unitful.uparse(repr(a))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[13]:1

julia> Unitful.uparse(repr(b))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[14]:1

@sostock
Copy link
Collaborator

sostock commented May 20, 2021

I don’t think this is easily possible right now, since there is no way to recover the symbol that a unit is bound to just from its value.

For example, in UnitfulAtomic.jl there is a unit UnitfulAtomic.ħ_au that is printed as ħ. However, uparse("ħ") evaluates to Unitful.ħ (which is a quantity in J*s, not a unit), not UnitfulAtomic.ħ_au.

@MartinOtter
Copy link
Author

One way would be to provide a utility function that converts a unit to a parseable unit string, something like "parseableString(vunit)"

@MartinOtter
Copy link
Author

MartinOtter commented Mar 2, 2022

Any news here?
Note, whenever a unit needs to be stored on file (say JSON file), it needs to be converted to a string (and
when reading from file, the string needs to be converted from the string representation to the Unitful representation).
Any advice how to do this with the current Unitful release?

To repeat the most important information:
string(unit(v)) returns a string that cannot be parsed. In Julia this seems to be an unusual behavior because string(something) typically returns a string representation of something that can be again parsed by Julia.
Alternatively, a function should be provided to do this, e.g. uparse( parseableString(2.0u"m/s") ) == u"m/s".

My current bad implementation is

parseableString(var_with_unit) = replace(repr(unit(var_with_unit), context = Pair(:fancy_exponent,false)), " " => "*")

@sostock
Copy link
Collaborator

sostock commented Mar 5, 2022

One way would be to return a string of the actual representation of the unit:

julia> str = parseable(u"km^2") # this function does not exist yet
"FreeUnits{(Unit{:Meter, Dimensions{(Dimension{:Length}(1),)}()}(3, 2),), Dimensions{(Dimension{:Length}(2),)}(), nothing}()"

julia> Unitful.eval(Meta.parse(str))
km^2

julia> ans === u"km^2"
true

Right now, parsing such a string does work with Unitful.eval ∘ Meta.parse, but uparse would have to be adapted to accept it:

julia> uparse(str)
ERROR: ArgumentError: FreeUnits{(Unit{:Meter, Dimensions{(Dimension{:Length}(1),)}()}(3, 2),), Dimensions{(Dimension{:Length}(2),)}(), nothing} is not a valid function call when parsing a unit.
Only the following functions are allowed: [:*, :/, :^, :sqrt, :, :+, :-, ://]

@michikawa07
Copy link
Contributor

michikawa07 commented Sep 7, 2022

It seems that only show(x::Quantity, etc.) are defined on Unitful.jl, and string(x::Quantity) are not defined.
Implicitly, string calls print and print calls show, so string and show are currently identical.
(The show determines output to the REPL, etc.).

I am currently trying to create a package called UnitfulParsabelString.jl, which will allow me to add a parsable string without breaking Unitful by defining Unitful.string(x::Quantity).

julia> using Unitful, UnitfulParsableString # <- my package

julia> a = u"mg/dL/dL"
mg dL⁻²

julia> string(a)
"mg*dL^-2"

julia> string(a) |> uparse
mg dL⁻²

julia> b = u"angstrom^3"
ų

julia> string(b)
"Å^3"

julia> string(b) |> uparse
ų

julia> c = 1.0u"m*kg/s^2"
1.0 kg m s⁻²

julia> string(c)
"1.0(kg*m*s^-2)"

julia> string(c) |> uparse
1.0 kg m s⁻²

Is there a demand for this package?
And if so, what characteristics should it have?
(Currently, there are some bugs (it does not parse well when the exponent part is a fraction, i.e., u"m^(1/2)"), so it is not available to the public.)

@michikawa07
Copy link
Contributor

For example, in UnitfulAtomic.jl there is a unit UnitfulAtomic.ħ_au that is printed as ħ. However, uparse("ħ") evaluates to Unitful.ħ (which is a quantity in J*s, not a unit), not UnitfulAtomic.ħ_au.

This problem may be solved by specifying the unit_context argument to uparse(str; unit_context).

function uparse(str; unit_context=Unitful)
    ex = Meta.parse(str)
    eval(lookup_units(unit_context, ex))
end

@MartinOtter
Copy link
Author

It seems that only show(x::Quantity, etc.). are defined on Unitful.jl, and string(x::Quantity). are not defined. Implicitly, string calls print and print calls show, so string and show are currently identical. (The show determines output to the REPL, etc.).
...
julia> string(c)
"1.0(kgms^-2)"

julia> string(c) |> uparse
1.0 kg m s⁻²

Is there a demand for this package? And if so, what characteristics should it have? (Currently, there are some bugs (it does not parse well when the exponent part is a fraction, i.e., u"m^(1/2)"), so it is not available to the public.)

Yes, there is a demand for such a package (e.g. whenever something needs to be stored on file in a standardized format such as JSON). So, I highly appreciate if you can provide such a package (it would be even better, if it would be part of Unitful.jl).

All your examples looks good. I am just a bit surprised for:

julia> string(c)
"1.0(kgms^-2)"

that this can be parsed with uparse. I checked, and indeed this works with Unitful.jl.

@michikawa07
Copy link
Contributor

michikawa07 commented Sep 11, 2022

@MartinOtter You have encouraged me to complete the package to the point where I am satisfied with it.
This is in the process of registering it as an official package, but you are welcome to use it if you like.
https://github.com/michikawa07/UnitfulParsableString.jl

(@v1.8) pkg > add https://github.com/michikawa07/UnitfulParsableString.jl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants