dir/server: fix tree.ListDir and workaround for Glob#159
Closed
dir/server: fix tree.ListDir and workaround for Glob#159
Conversation
Change-Id: Ib6b8cf777d48e3420ec5f0881f629b27a99c3c53 Reviewed-on: https://upspin-review.googlesource.com/9980 Reviewed-by: Eric Grosse <grosse@gmail.com>
…th names - but don't use it yet Given certain references, the existing code could try to create files with empty names or names that collided with directories. Add a new version, not yet enabled, that fixes those bugs. Also make it a little better at scaling by providing more levels of fanout. Refactor the package to put the path handling into a new internal package that contains both the old and new encodings. This will make it easy to write a conversion tool that will update an existing storage tree. Add a test for the new encoding. Change-Id: Ia6f9ed92499ea231164ce99876b9bf83cd727424 Reviewed-on: https://upspin-review.googlesource.com/9900 Reviewed-by: Andrew Gerrand <adg@golang.org>
…ew-style one This tool can be used to help move a tree to the new encoding. Tested by creating a tree and copying it. Once this lands, the package can be updated to recommend running the convert tool if the tree is in the old format. Change-Id: Id8e25fdb7bfc71ce191d3ed79ba1422e32ccd809 Reviewed-on: https://upspin-review.googlesource.com/9901 Reviewed-by: Andrew Gerrand <adg@golang.org>
Also add a test to see that the tree is using the new encoding, and scream if not. Change-Id: I0a9efcd82ca9583221e550e4c66a2f6fbbce6944 Reviewed-on: https://upspin-review.googlesource.com/9960 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Eduardo Pinheiro <epinheiro@gmail.com>
There's no concept of a bucket name when using on-disk storage. Change-Id: I3e194f1c4cc6160962857e9f9e96fda9c72820e3 Reviewed-on: https://upspin-review.googlesource.com/10000 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: Ie38171ec735e72b2c8d9ada042c5c8e62fc9b472 Reviewed-on: https://upspin-review.googlesource.com/10001 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Eduardo Pinheiro <epinheiro@gmail.com> Reviewed-by: David Presotto <presotto@gmail.com>
…g immediately when open count goes to 0 Change-Id: I06923bb38b4a48908fd35484f03245381ffc3d3e Reviewed-on: https://upspin-review.googlesource.com/10041 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Eric Grosse <grosse@gmail.com>
Change-Id: I3677e453699276ac9f88ddff3a502195e05bd15f Reviewed-on: https://upspin-review.googlesource.com/10040 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: Ia5412b847afc3f00415b1bafcee82620101b3fee Reviewed-on: https://upspin-review.googlesource.com/10060 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
We already have upspin.Config. Let's use a name unique to upbox. Fix #386 Change-Id: Iefed9a66a0ec02c2d01c2942af59a59d718c2107 Reviewed-on: https://upspin-review.googlesource.com/10061 Reviewed-by: Rob Pike <r@golang.org>
This leaves exp/cmd/upbox a simple wrapper around the upbox package. The plan is to extend upbox.Schema so that it can be used in the cmd/upspin tests, and more. Change-Id: I2b2e96365e5396ab0d83d8d85dfcbec3d428cdf0 Reviewed-on: https://upspin-review.googlesource.com/10062 Reviewed-by: Rob Pike <r@golang.org>
This was a bit of a mess. This change is a step toward cleaning it up a bit. It's likely that some of the method arguments may become fields of Schema, but not yet. Change-Id: I03b246ff95eeba96ad75d11bbe5d2f166fd75208 Reviewed-on: https://upspin-review.googlesource.com/10063 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: Ib3fe4a6cc9fd12c522d969becdbbd8c8ea3776c0 Reviewed-on: https://upspin-review.googlesource.com/10064 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: Ie32155cce37e6edf9f4fc358ecbfebef539a4f1e Reviewed-on: https://upspin-review.googlesource.com/10066 Reviewed-by: Rob Pike <r@golang.org>
This will make it easier for callers of the forthcoming Start and Stop methods to locate the user config files. Change-Id: Ieb320f3d96382a3f0bfca3ee63e3fbf427bd07fc Reviewed-on: https://upspin-review.googlesource.com/10067 Reviewed-by: Rob Pike <r@golang.org>
This noise clutters up every upbox run. Change-Id: Ibbc368b690bcb91a11fe73d7591c7ea6781561ed Reviewed-on: https://upspin-review.googlesource.com/10068 Reviewed-by: Rob Pike <r@golang.org>
Re-apply CL/8673 with fix for broken test. Change 1 of 2. Prepare the ground work for log rotation by adding functions to manipulate rotated log names and a subdirectory where they will exist. Backward compatibility is ensured by adding an offset 0 hard link to the current log file. Refactored Log into Writer and Reader, which is what the Clone method was effectively doing. Next CL will do the actual rotation. Added a test to ensure progress was made. Test will be updated in the next CL. Also cleaned up current test setup. Updates #184 Change-Id: Ifdd372bcb6808859d02b759cb2e8889ac9c44bf7 Reviewed-on: https://upspin-review.googlesource.com/9940 Reviewed-by: David Presotto <presotto@gmail.com>
Also export a Config method to get the path to a user's config file. Change-Id: Ibaf10cfaaeaab7f6e59c2f3748936393a99a1e30 Reviewed-on: https://upspin-review.googlesource.com/10069 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: I0ea68ca2d16cf480cff3fb2cd8d47b1d30a93d88 Reviewed-on: https://upspin-review.googlesource.com/10070 Reviewed-by: Rob Pike <r@golang.org>
Most users aren't interested in info-level log messages. Change-Id: I7f27d6b398f5b13b9ff25696aa09385fa3caf163 Reviewed-on: https://upspin-review.googlesource.com/10071 Reviewed-by: Rob Pike <r@golang.org>
Playing with disconnected service, I noticed that the logic is broken for retrying a Watch. Change-Id: I9dc3b858fd2a17df3525657d154b4743553cdbe0 Reviewed-on: https://upspin-review.googlesource.com/10080 Reviewed-by: Rob Pike <r@golang.org>
One downside from the previous CL to prepare the logs for rotation is that new roots are not compatible with the old server (without the previous CL). So, reverting to and old server after creating a new user with the new server makes the log start from scratch and unflushed direntries will be lost. With this change, we make sure we have an old-style log file around in case the server binary gets reverted for some reason. Note that this scenario of running an old binary after a new root with a new binary has been made is unlikely. Moreover, it will only be revertible until the first log rotation happens (which currently is never). Tested on upspin-test. Change-Id: I48f747e9cf660cfa0b3de5fe9ceaf59b2f288427 Reviewed-on: https://upspin-review.googlesource.com/8804 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: David Presotto <presotto@gmail.com>
Let those so inclined enable extra security checks. Change-Id: I15013e89cdac83ff8aa2ff30c3e75b192b792751 Reviewed-on: https://upspin-review.googlesource.com/10100 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: David Presotto <presotto@gmail.com>
When logs exceed a certain size (currently 100MB), create a new log file and leave the previous ones intact (for now). Updates #184 Change-Id: I88f0316a4f9b8748d103c1abb05b2296508103c1 Reviewed-on: https://upspin-review.googlesource.com/10160 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
A 1MB DirEntry is not only possible, but likely in real systems. Allow a 64MB entry for now until we properly resolve issue #347. Updates #347. Change-Id: Iddb3f120b793b5fd4e48b0b0b80be7b629cae855 Reviewed-on: https://upspin-review.googlesource.com/10120 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
Do some refactoring to make it easier to write upspin tests: - provide overrides for stdin, stdout, stderr. - allow commands to be run one at a time from external control, treating the command like a package. This is the purpose of the new Setup and Run functions. Finally, provide a trivial new test to demonstrate the workings. This will expand considerably as real tests are written. The test uses upbox to put the environment together, then runs a single "user" command to test it's working. Change-Id: I84a9e880970169be7bb1de5c732c80ab8e05450c Reviewed-on: https://upspin-review.googlesource.com/10073 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Eduardo Pinheiro <epinheiro@gmail.com>
The original DirServer is allocated with server.New(). Then clients can dial the server and create a new instance. Not all clients close their instances, which makes garbage collection difficult in tests. This change makes it so when the original undialed server is closed, all trees are flushed and closed. In tests, this means TMPDIR/testenv-dirserver* directories are not left behind. Also fixed tests to properly call env.Exit() if they call env.New(). Fixes #392 Change-Id: I1d78329776bbaef13df263bda2840835855aefc3 Reviewed-on: https://upspin-review.googlesource.com/10180 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: David Presotto <presotto@gmail.com>
flags.Parse uses the command line itself. Here we provide a way to pass in a different command line. This will make some testing easier and might be useful in other cases. Change-Id: Id420e4bac1aa10b026ed69700659b1e354c28a6f Reviewed-on: https://upspin-review.googlesource.com/10072 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Eduardo Pinheiro <epinheiro@gmail.com>
See if we can spot the reason why this test is flaky (and only on the CI). Updates #389 Change-Id: Ib7db3a5886c692c8eadbff5ac17cbe0494e4a1f1 Reviewed-on: https://upspin-review.googlesource.com/10200 Reviewed-by: David Presotto <presotto@gmail.com>
Previously, WhichAccesses were remembered only for the target file and sometimes its directory. Now we only save it for the directory. When looking up a cached answer for WhichAccess, we now walk up the tree looking for a relevant Access file, not just looing for one cached for the target file or its directory. Change-Id: I9e10aeb1fcb349d9c0749455250b6087b0f442df Reviewed-on: https://upspin-review.googlesource.com/10240 Reviewed-by: Eric Grosse <grosse@gmail.com> Reviewed-by: Eduardo Pinheiro <epinheiro@gmail.com>
On platforms where a user does not have a home directory (such as Apple's iOS) the flags package would raise a panic at init time. This change removes the panic and instead uses "." as the home directory when setting flag defaults on such systems. Fixes #364 Change-Id: I4b11808bb84ddd891f98d93fac10956d27df899f Reviewed-on: https://upspin-review.googlesource.com/10260 Reviewed-by: Eric Grosse <grosse@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
Fixes #396 Change-Id: Ie495c8f8d470701f9f4ec438f5fc9008845e96ec Reviewed-on: https://upspin-review.googlesource.com/10261 Reviewed-by: Rob Pike <r@golang.org>
This is a simple read-only Upspin server that serves GitHub issues. The resulting tree looks like this (issue 1 is closed and 2 is open): user@example.com/owner/repo/all/1 user@example.com/owner/repo/all/2 user@example.com/owner/repo/closed/1 (link to all/1) user@example.com/owner/repo/open/2 (link to all/2) Change-Id: Iac2d03b620f5bc63e24fd000792ced776d8618af Reviewed-on: https://upspin-review.googlesource.com/10220 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: I21469ef0bab792993a6847063d86871f005a09fe Reviewed-on: https://upspin-review.googlesource.com/10262 Reviewed-by: Andrew Gerrand <adg@golang.org>
The GOPATH environment variable will now default to $HOME/go if it is unset. Use the GOPATH from the go/build package's default environment to inherit this property. Change-Id: I86a20249b7c357abee1870369f3b87279a5e1f7b Reviewed-on: https://upspin-review.googlesource.com/10263 Reviewed-by: Rob Pike <r@golang.org>
Make this feature work correctly on systems with multiple GOPATHs. Fix #397 Change-Id: I72a69aca12a43cca61e8081f50af782726a02603 Reviewed-on: https://upspin-review.googlesource.com/10280 Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
The user advice for recreating keys should note the curve, in case the user is not using the default. Fix #399 Change-Id: I3b6bc1bc27118797c2f2c86a4b8a597f117366d1 Reviewed-on: https://upspin-review.googlesource.com/10300 Reviewed-by: Eduardo Pinheiro <edpin@google.com> Reviewed-by: David Presotto <presotto@gmail.com>
Removed most of the unused code found by an automated tool (the 'unused' tool from https://github.com/dominikh/go-tools). I did not remove some unused fields in structs and unused methods because they need more investigation (maybe they were meant to be used or will in the future). Change-Id: Ibe048ce22624b609a4067479766d37cceaf6243d Reviewed-on: https://upspin-review.googlesource.com/10320 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
Switch to using an external package for gzipped responses, as the package does more than what our previous implementation did; in particular, it detects the content type and sets the appropriate header before writing the response. Our implementation was broken in that the built-in content type sniffing was always detecting the gzipped response, which was almost always wrong. Change-Id: Id36adf2ea9f782325ad19ef30b042ba0792a2796 Reviewed-on: https://upspin-review.googlesource.com/10380 Reviewed-by: Rob Pike <r@golang.org>
…erutil/web The new serverutil/web package provides an http.Handler that serves content from Upspin space. Change-Id: I56bc4926b6be820251d18ed0acd17bddb23aedb4 Reviewed-on: https://upspin-review.googlesource.com/10341 Reviewed-by: Rob Pike <r@golang.org>
This permits us to serve Upspin release binaries from upspin.io. Change-Id: Ibf044c52cc05163ad9fcc9130b53346857022b2f Reviewed-on: https://upspin-review.googlesource.com/10342 Reviewed-by: Rob Pike <r@golang.org>
Avoid copying lock pointers by simply storing the parent Writer. This was left over from Dave's review that I'm now addressing. Change-Id: I61b18a6f1d55df0ff3517daad9b3019254c3688b Reviewed-on: https://upspin-review.googlesource.com/10401 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Rob Pike <r@golang.org>
ListUsers with an arbitrary pattern was dangerous because it gave the caller too much power to shoot him/herself on the foot (put another way, it required too much internal knowledge). This change exposes two simpler functions that do not require the caller to form arbitrary globs. They should be easier to maintain if/when the underlying log naming format changes. Change-Id: I56a52c83759f9c06514f4e6b7b29e060c399489a Reviewed-on: https://upspin-review.googlesource.com/10402 Reviewed-by: Eric Grosse <grosse@gmail.com> Reviewed-by: Andrew Gerrand <adg@golang.org>
Change-Id: Ie6331116fc05c52eb7685dd4f382e99593b1aa71 Reviewed-on: https://upspin-review.googlesource.com/10421 Reviewed-by: Rob Pike <r@golang.org>
Fixes #403 Thanks to @shurcooL for the repro case. Change-Id: I3d0dc3cf73800abfbfa1b1bf4d1eb9d4663317dd Reviewed-on: https://upspin-review.googlesource.com/10400 Reviewed-by: Rob Pike <r@golang.org>
The serverutil.ListFunc docs say:
It should return an ErrFollowLink error iff the given path name is a
link. In that one case, it should also return only the DirEntry for
that path.
The dir/server implementation was not doing this (dir/inprocess is OK).
Also add a test to package test.
Change-Id: Id7156a12797bdcb6a0f24c07da697ec13b22438d
Reviewed-on: https://upspin-review.googlesource.com/10440
Reviewed-by: Rob Pike <r@golang.org>
Change-Id: I43e8ad76236835395b206b48c947839c79952e24 Reviewed-on: https://upspin-review.googlesource.com/10361 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Bons <andybons@google.com>
Windows users typically prefer zip files over tar.gz files. Change-Id: Iebcf8318ccfb40567173ac572566bde82101f0b7 Reviewed-on: https://upspin-review.googlesource.com/10460 Reviewed-by: Rob Pike <r@golang.org>
… Access file Since WhichAccess returns the DirEntry for the Access file, no reason to use Get with the file name. Change-Id: Ibd8f591f06ea4d79c804f056e7864ae8da14858d Reviewed-on: https://upspin-review.googlesource.com/10501 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
It is confusing for people to see a message "Configuration file not found." A few have assumed that it is an error message, when in fact it is the desired state for the upspinserver to be in at this point in the process. Change-Id: Icf5eb66c7814a3705874490e0caf7626786667ac Reviewed-on: https://upspin-review.googlesource.com/10520 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org>
Now that upbox is used by the tests in cmd/upspin, it should no longer be considered experimental. Change-Id: Ic52f4d63b801152b43dde29f62e676d2891e5261 Reviewed-on: https://upspin-review.googlesource.com/10461 Reviewed-by: Rob Pike <r@golang.org>
Change-Id: I79c7cdecc45154ad4e94346042412d8d4cd2fae0 Reviewed-on: https://upspin-review.googlesource.com/10522 Reviewed-by: Rob Pike <r@golang.org>
macOS is the correct current spelling [0][1] of the operating system name. MacOS visually resembles "Mac OS", which is the spelling of "Classic" Mac OS operating system from 1984 to 2001 [2]. References: [0] https://www.apple.com/macos/sierra/ [1] https://en.wikipedia.org/wiki/MacOS#macOS [2] https://en.wikipedia.org/wiki/Classic_Mac_OS Change-Id: I83b7fc145e84f31ba942602a607b944b4e4f4bc4 Reviewed-on: https://upspin-review.googlesource.com/10502 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Rob Pike <r@golang.org>
Made a Fatal be an Errorf instead and added some context to help with debugging. The previous instrumentation was aborting the test with Fatal, but we can keep going to see what happens in the second pass of the loop, which should either provide us with more events (and if so, these would be spurious and could lead to the actual issue) or not. Updates #389 Change-Id: Ic06a6d5fa9ed71d9b02bfbd917d1019dc93ee7e4 Reviewed-on: https://upspin-review.googlesource.com/10500 Reviewed-by: David Presotto <presotto@gmail.com> Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
AugieBot
force-pushed
the
I3023298df11c4e0a571a45d9b2c7eb4d2acb191d
branch
from
4e164e1 to
f154afd
Compare
There is no need to do a lookup before tree.List. But tree.List had a bug processing links and is now fixed. Change-Id: I3023298df11c4e0a571a45d9b2c7eb4d2acb191d
AugieBot
force-pushed
the
I3023298df11c4e0a571a45d9b2c7eb4d2acb191d
branch
from
f154afd to
75063d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automatically created pull request. Do not review or merge this PR.