fix #61: implement decodeDelimitedMessageH (decoding from a file handle) #324

ulysses4ever · 2019-06-09T20:30:55Z

No description provided.

googlebot · 2019-06-09T20:30:58Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

judah · 2019-06-10T01:43:18Z

Thank you for the patch! I'll take a look once the CLA is signed. Let me know if you have any questions about getting it working.

mboes · 2019-06-10T07:15:30Z

@ulysses4ever as been added to Tweag's google-contributors@ whitelist. So he has transitively signed the CLA.

googlebot · 2019-06-10T13:17:14Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

judah

Thank you for the patch! The overall approach looks reasonable; I've suggested some minor changes.

judah · 2019-06-10T13:29:08Z

proto-lens/src/Data/ProtoLens/Encoding/Bytes.hs

+getVarIntH h = do
+    buf <- malloc
+    let loopStart !s !n = do
+          _ <- hGetBuf h buf 1


You should check the result of hGetBuf. It might return a value of zero at EOF, in which case b would be undefined. At worst case, that would mean this could loop forever.

Good point! What would be a reasonable way to fail gracefully in case hGetBuf returns 0?

Hm, after thinking about it: since the function that ultimately calls this one returns IO (Either String ...), you should probably return a similar type from this function: IO (Either String Word64).

Maybe it would make sense to simplify the internal error-passing via ExceptT String IO. Up to you.

judah · 2019-06-10T13:32:17Z

proto-lens-tests/package.yaml

@@ -209,6 +209,17 @@ tests:
      - Proto.Any
      - Proto.Any_Fields

+  decode_delimited:


Nit: please follow the convention of the other tests and name this "decode_delimited", and name the corresponding source file "decode_delimited_test.hs".

Sorry, I didn't get: is the name of the test wrong too? The name of the file I will fix, thanks!

Yeah, by convention the test names in package.yaml also have "_test" at the end.

judah · 2019-06-10T13:34:48Z

proto-lens-tests/tests/decode_delimited.hs

+          h <- openBinaryFile filename ReadMode
+          any1 <- decodeDelimitedMessageH h :: IO (Either String Foo)
+          hClose h
+          removeIfExists filename


When would we not expect the file to exist? Would it make the test simpler/more robust if we just threw an error if the file unintentionally disappeared?

That makes sense. Do you want me to leave just the call to removeFile (which might throw) without additional bookkeeping?

withSystemTempFile should remove the file automatically, so I believe you could just delete the call.

judah · 2019-06-10T13:36:57Z

proto-lens-tests/tests/decode_delimited.hs

+    [ testCase "buildDelimited/decodeDelimited" $ do
+          let foo = defMessage & a .~ 42 & b .~ "hello" :: Foo
+          let bs = runBuilder . buildMessageDelimited $ foo
+          B.writeFile filename bs


Let's use the temporary package to avoid writing files in the current directory, which would be the proto-lens-tests source directory in this case.
http://hackage.haskell.org/package/temporary-1.3/docs/System-IO-Temp.html#v:withSystemTempFile

judah · 2019-06-10T13:38:32Z

proto-lens/src/Data/ProtoLens/Encoding.hs

@@ -7,8 +7,11 @@ module Data.ProtoLens.Encoding (
    -- ** Delimited messages
    buildMessageDelimited,
    parseMessageDelimited,
+    decodeDelimitedMessageH,


I think we should try to mirror the naming scheme of the other functions:

decodeMessage, decodeMessageDelimited, decodeMessageDelimitedH.

Does that sound reasonable to you?

Ouch, silly me! I can't believe I got this wrong. Will fix.

ulysses4ever · 2019-06-11T14:50:12Z

I tried to address all your comments, thanks a lot for reviewing! Waiting for more comments!

judah · 2019-06-13T15:07:41Z

proto-lens/src/Data/ProtoLens/Encoding.hs

@@ -47,3 +52,10 @@ parseMessageDelimited = do
    len <- Bytes.getVarInt
    bytes <- Bytes.getBytes $ fromIntegral len
    either fail return $ decodeMessage bytes
+
+-- | Same as @decodeMessage@ but for delimited messages read through a Handle
+decodeMessageDelimitedH :: Message msg => Handle -> ExceptT String IO msg


Hm, my original suggestion had been to keep ExceptT internal to the implementation, and still return IO (Either String msg) from this function.

Now that I see this, I'm on the fence which way is better, but leaning towards plain IO. I think ExceptT could make sense if we had several different functions we wanted to compose, but I'm not sure we'll add anything beyond this one.

What do you think? Would ExceptT simplify the code you plan to use this with, or would you end up just unwrapping it with runExceptT when you used it?

I agree with you: returning IO of Either feels more natural in the current interface than returning ExceptT. Let me implement that.

ulysses4ever · 2019-06-14T13:47:13Z

Thanks a lot! I'd appreciate if you could release it any time soon (but no rush, of course).

judah · 2019-06-15T15:09:24Z

@ulysses4ever I've released proto-lens-0.5.1.0 containing this change.

ulysses4ever · 2019-06-15T17:03:27Z

@judah thanks a lot!

…oogle#61 (google#324)

googlebot added the cla: no label Jun 9, 2019

googlebot added cla: yes and removed cla: no labels Jun 10, 2019

judah requested changes Jun 10, 2019

View reviewed changes

judah reviewed Jun 13, 2019

View reviewed changes

implement decodeMessageDelimitedH (decoding from a file handle); fix #61

2c10c31

judah approved these changes Jun 14, 2019

View reviewed changes

judah merged commit 5eb4ea1 into google:master Jun 14, 2019

ulysses4ever deleted the add-decode-delimited-handle branch June 14, 2019 13:49

ylecornec pushed a commit to ylecornec/proto-lens that referenced this pull request Feb 19, 2024

implement decodeMessageDelimitedH (decoding from a file handle); fix g…

fa17a6c

…oogle#61 (google#324)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix #61: implement decodeDelimitedMessageH (decoding from a file handle) #324

fix #61: implement decodeDelimitedMessageH (decoding from a file handle) #324

ulysses4ever commented Jun 9, 2019

googlebot commented Jun 9, 2019

judah commented Jun 10, 2019

mboes commented Jun 10, 2019

googlebot commented Jun 10, 2019

judah left a comment

judah Jun 10, 2019

ulysses4ever Jun 10, 2019

judah Jun 10, 2019

judah Jun 10, 2019

ulysses4ever Jun 10, 2019

judah Jun 10, 2019

judah Jun 10, 2019

ulysses4ever Jun 10, 2019 •

edited

Loading

judah Jun 10, 2019

judah Jun 10, 2019

judah Jun 10, 2019

ulysses4ever Jun 10, 2019

ulysses4ever commented Jun 11, 2019

judah Jun 13, 2019

ulysses4ever Jun 13, 2019

ulysses4ever Jun 13, 2019

ulysses4ever commented Jun 14, 2019

judah commented Jun 15, 2019

ulysses4ever commented Jun 15, 2019

fix #61: implement decodeDelimitedMessageH (decoding from a file handle) #324

fix #61: implement decodeDelimitedMessageH (decoding from a file handle) #324

Conversation

ulysses4ever commented Jun 9, 2019

googlebot commented Jun 9, 2019

What to do if you already signed the CLA

Individual signers

Corporate signers

judah commented Jun 10, 2019

mboes commented Jun 10, 2019

googlebot commented Jun 10, 2019

judah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ulysses4ever Jun 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ulysses4ever commented Jun 11, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ulysses4ever commented Jun 14, 2019

judah commented Jun 15, 2019

ulysses4ever commented Jun 15, 2019

ulysses4ever Jun 10, 2019 •

edited

Loading