proposal: io/fs: add fs.Files() iterator

[myaaaaaaaaa]

Proposal Details

A previous proposal for walking through directory trees can be found here:

• proposal: io/fs,path/filepath: add iterator form of WalkDir #64341

This proposal aims for a simpler interface, making a deliberate decision to focus on the use case of reading file contents.

Signature

package fs

func Files(fsys FS, err *error) iter.Seq2[string, File]

Usage

func main() {
	var err error
	for pathName, file := range fs.Files(os.DirFS("."), &err) {
		if !strings.HasSuffix(pathName, ".json") {
			// `file` is automatically opened on the first call to Read(),
			// so skipping here doesn't waste any syscalls/network traffic.
			continue
		}

		// errors are automatically collected at the end
		json, _ := io.ReadAll(file)
		fmt.Println(json)

		// `file` is automatically closed after each iteration
	}

	// loop automatically terminates early if any error occurs,
	// including in `file` operations.
	if err != nil {
		fmt.Println(err)
	}
}

[gabyhelp]

Related Issues and Documentation

• proposal: io/fs,path/filepath: add iterator form of WalkDir #64341

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[qiulaidongfeng]

Why is the function declaration not func All(fsys FS, err *error) iter.Seq2[string, File]

[ianlancetaylor]

I think this would be better handled as part of the discussion on #64341, rather than splitting the discussion of iterators over file systems into two separate proposals.

[myaaaaaaaaa]

Why is the function declaration not func All(fsys FS, err *error) iter.Seq2[string, File]

I went back and forth on this for a bit, but applied this suggestion in the end.

Note that because errors are automatically collected by the iterator, the File being yielded will have to be a wrapper - it can't just pass through the one returned by fsys directly.

I think this would be better handled as part of the discussion on #64341, rather than splitting the discussion of iterators over file systems into two separate proposals.

The design goals seemed to differ enough to me - #64341 attempts to preserve all of the functionality of WalkDir (such as DirEntrys and SkipDir), while this proposal deliberately sacrifices those for the sake of making it easier to work with file contents.

Notably, this would have been proposed as fs.GlobSeq() if double star matches ("**/*.json") were supported

[ianlancetaylor]

Let me put it this way: we are only going to implement one iterator over the files in an fs.FS. I think it would be better to discuss what we should implement in a single place. That includes what functionality the iterator should implement.

[myaaaaaaaaa]

Closing in favor of #64341