Walk function seems to only pack the first alphabetical path

[paganotoni]

Hi @markbates 👋!

Context

I've been trying to use pkger as a replacement for packr2 in a small tool I've been writing (github.com/paganotoni/admon).

The things I need to replace initially are the assets and templates Boxes. To do so, I'm walking both paths with pkger and filling in a pckd.MemoryBox with those files so buffalo can use them.

var (
        assetsBox    = packd.NewMemoryBox()
	templatesBox = packd.NewMemoryBox()
)
func init() {
	logrus.Info("Reading templates")
	err := pkger.Walk("/web/templates", func(path string, info os.FileInfo, err error) error {
		logrus.Info(path)

		if err != nil {
			return err
		}

		if info.IsDir() {
			return nil
		}

		file, err := pkger.Open(path)
		if err != nil {
			return err
		}

		data, err := ioutil.ReadAll(file)
		if err != nil {
			return err
		}

		return templatesBox.AddBytes(path, data)
	})

	if err != nil {
		logrus.Error(err)
	}

	logrus.Info("Reading assets")
	err = pkger.Walk("/web/public", func(path string, info os.FileInfo, err error) error {
		logrus.Info(path)

		if err != nil {
			return err
		}

		if info.IsDir() {
			return nil
		}

		file, err := pkger.Open(path)
		if err != nil {
			return err
		}

		data, err := ioutil.ReadAll(file)
		return assetsBox.AddBytes(path, data)
	})

	if err != nil {
		logrus.Error(err)
	}

	renderEngine = render.New(render.Options{
		HTMLLayout:   "admon.plush.html",
		TemplatesBox: templatesBox,

		AssetsBox: assetsBox,
		Helpers:   helpers,
	})
}

The problem

The problem i'm facing is that pkger seems to only walk through the first path alphabetically. And is ignoring the second one as the following logs show:

time="2019-10-23T08:21:47-05:00" level=info msg="Reading templates"
time="2019-10-23T08:21:47-05:00" level=info msg="Reading assets"
time="2019-10-23T08:21:47-05:00" level=info msg="github.com/paganotoni/admon:/web/public"
time="2019-10-23T08:21:47-05:00" level=info msg="github.com/paganotoni/admon:/web/public/assets"
time="2019-10-23T08:21:47-05:00" level=info msg="github.com/paganotoni/admon:/web/public/assets/admin.e4904589a02627deb099.css"
time="2019-10-23T08:21:47-05:00" level=info msg="github.com/paganotoni/admon:/web/public/assets/admin.f11b630e4512aaa573d4.js"
time="2019-10-23T08:21:47-05:00" level=info msg="github.com/paganotoni/admon:/web/public/assets/manifest.json"

[markbates]

Can you run pkger parse and pkger list and add the output?

[paganotoni]

Sure, parse:

{
 ".": [
  {
   "file": {
    "Abs": "/Users/paganotoni/go/src/github.com/paganotoni/admon/admon.go",
    "Path": {
     "pkg": "",
     "name": ""
    },
    "Here": {
     "Dir": "$GOPATH/src/github.com/paganotoni/admon",
     "ImportPath": "github.com/paganotoni/admon",
     "Module": {
      "Dir": "$GOPATH/src/github.com/paganotoni/admon",
      "GoMod": "$GOPATH/src/github.com/paganotoni/admon/go.mod",
      "GoVersion": "1.13",
      "Main": true,
      "Path": "github.com/paganotoni/admon"
     },
     "Name": "admon"
    }
   },
   "pos": 17637,
   "type": "pkger.Walk",
   "value": "/web/templates"
  },
  {
   "file": {
    "Abs": "/Users/paganotoni/go/src/github.com/paganotoni/admon/admon.go",
    "Path": {
     "pkg": "",
     "name": ""
    },
    "Here": {
     "Dir": "$GOPATH/src/github.com/paganotoni/admon",
     "ImportPath": "github.com/paganotoni/admon",
     "Module": {
      "Dir": "$GOPATH/src/github.com/paganotoni/admon",
      "GoMod": "$GOPATH/src/github.com/paganotoni/admon/go.mod",
      "GoVersion": "1.13",
      "Main": true,
      "Path": "github.com/paganotoni/admon"
     },
     "Name": "admon"
    }
   },
   "pos": 18087,
   "type": "pkger.Walk",
   "value": "/web/public"
  }
 ]
}

List:

github.com/paganotoni/admon
 > github.com/paganotoni/admon:/web/templates
 > github.com/paganotoni/admon:/web/templates/admon.plush.html
 > github.com/paganotoni/admon:/web/templates/dashboard.html
 > github.com/paganotoni/admon:/web/templates/partials
 > github.com/paganotoni/admon:/web/templates/partials/_flash.plush.html
 > github.com/paganotoni/admon:/web/templates/partials/_header.plush.html
 > github.com/paganotoni/admon:/web/templates/partials/_navbar.plush.html
 > github.com/paganotoni/admon:/web/templates/resource
 > github.com/paganotoni/admon:/web/templates/resource/_column_header.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/_form.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/_summary.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/edit.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/index.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/new.plush.html
 > github.com/paganotoni/admon:/web/templates/resource/show.plush.html
 > github.com/paganotoni/admon:/web/public
 > github.com/paganotoni/admon:/web/public/assets
 > github.com/paganotoni/admon:/web/public/assets/admin.e4904589a02627deb099.css
 > github.com/paganotoni/admon:/web/public/assets/admin.f11b630e4512aaa573d4.js
 > github.com/paganotoni/admon:/web/public/assets/manifest.json

[paganotoni]

Update: This seems to be something more general. I tried to change the way I load files by doing Walk only for templates and then use Open to one-by-one add the assets.

	templatesBox = packd.NewMemoryBox()
        // The walker func is the same as the one I had before but extracted for reusability purposes.
	err := pkger.Walk("/web/templates/", walker("/web/templates", templatesBox))
       
	if err != nil {
		logrus.Error(err)
	}

	assetsBox = packd.NewMemoryBox()
	file, err := pkger.Open("github.com/paganotoni/admon:/web/public/assets/manifest.json")
	data, err := ioutil.ReadAll(file)
	if err != nil {
		logrus.Error(err)
	}

	assetsBox.AddBytes("manifest.json", data)

The output I get from this is it seems the templates folder is no longer being added to the packd files.

[markbates]

Can you try and simplify the example? Remover packr etc... and write a simple program that just walks those two and prints them out? There are currently too many “variables” in the example.

…

-- Mark Bates

On Oct 23, 2019, at 12:02 PM, Antonio Pagano ***@***.***> wrote:  Update: This seems to be something more general. I tried to change the way I load files by doing Walk only for templates and then use Open to one-by-one add the assets. templatesBox = packd.NewMemoryBox() err := pkger.Walk("/web/templates/", walker("/web/templates", templatesBox)) if err != nil { logrus.Error(err) } assetsBox = packd.NewMemoryBox() file, err := pkger.Open("github.com/paganotoni/admon:/web/public/assets/manifest.json") data, err := ioutil.ReadAll(file) if err != nil { logrus.Error(err) } assetsBox.AddBytes("manifest.json", data) The output I get from this is it seems the templates folder is no longer being added to the packd files. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[paganotoni]

Sure. Let me try and write something

[paganotoni]

Wow @markbates that was a great suggestion. I'm thinking this has something to do with packer/packd. When i do the same without using packd it seems to work. closing this issue.