Ability To Turn Off Formatting For Subdirectory #873

samdeane · 2024-11-07T10:45:34Z

Draft implementation of #870.

This implementation adds a check for a .swift-format-ignore file, in the same places that a .swift-format file could be.

If present, this file potentially contains gitignore-style rules describing which files to ignore.

Currently the only rule supported is *, to ignore everything in the containing directory and any subdirectories. Later, full pattern-matching rules could be implemented, if desired. For now, we simply check that the content of the file is *, to ease any future migration problems.

The presence of .swift-format-ignore is checked first at any given directory level. Even if a .swift-format file also exists at the same level, the ignore file takes precedence.

Motivation

The use case which motivated this change is a scenario where my workspace has a .swift-format file, and I've turned on format-on-save, but I also have submodules containing third-party code.

If I make a small edit to a file in a submodule, it is formatted using the global workspace settings. This generates a large diff unrelated to my edit, as the submodule is hand-formatted using different rules.

I want to be able to disable formatting for specific submodules on my local machine, whilst retaining formatting for the rest of the workspace.

allevato

Thanks for looking into this! I think it's a useful feature to have. Some high-level comments:

I think the name of the file should be .swift-format-ignore, which aligns nicely with the name used in comments that the formatter supports, and also similar tools like .gitignore and .prettierignore.
Lowering this down into the SwiftFormatter and SwiftLinter APIs feels like the wrong approach, because it means the tool is still doing all the work of iterating over and parsing all the files that are going to be ignored anyway.

For the second point, raising this check into the frontend seems more appropriate. Maybe FileIterator should be responsible for looking for the .swift-format-ignore file in the directory or its parents, and if one is found, it doesn't recurse into the directory further? That may end up being simpler to implement (you don't have to touch the configuration at all).

It means that somebody using the SwiftFormatter/SwiftLinter APIs directly wouldn't get that logic, but I think that's fine; those APIs are meant to be used by clients who want to format something directly and supplying a configuration in memory, and they're not interested in the search logic used by the frontend.

samdeane · 2024-11-07T14:11:21Z

I think the name of the file should be .swift-format-ignore

Yes, makes sense 👍

samdeane · 2024-11-07T14:14:16Z

For the second point, raising this check into the frontend seems more appropriate. Maybe FileIterator should be responsible for looking for the .swift-format-ignore

I think I was fixated on the idea that the ignoring mode needed to be configurable from within a .swift-format, as well as by the presence of a .swift-format-ignore file. Having done it though, I struggle to see the utility of that setting.

I'll try your plan instead.

samdeane · 2024-11-07T14:50:09Z

Ok, I got rid of the setting and moved the check into FileIterator's directory handling.

It works well for input directories, and for subdirectories reached via recursion, but doesn't work for input files (e.g: in the case where individual file paths are specified on the command line directly).

For input files, we need to perform a check for .swift-format-ignore in the same directory as the file. Putting that into the iterator's next() feels wrong as it's not needed in most cases. My inclination is to filter the input urls in init, and perform this test on any that are files.

allevato · 2024-11-07T14:55:56Z

For input files, we need to perform a check for .swift-format-ignore in the same directory as the file. Putting that into the iterator's next() feels wrong as it's not needed in most cases. My inclination is to filter the input urls in init, and perform this test on any that are files.

That makes the most sense to me.

samdeane · 2024-11-07T15:26:37Z

Ok, 'tis a little closer now.

One question is what it should do if --configuration <file> is passed in.

The README still says that the filesystem won't be searched; which is true for .swift-format, but no longer true for .swift-format-ignore.

The path of least resistance is to clarify the README, but that depends whether that's the behaviour we want.

Alternatively I'll need to add a flag to FileIterator to indicate whether or not it should check for the ignore file. Which is perfectly doable, of course.

allevato · 2024-11-07T15:42:04Z

One question is what it should do if --configuration <file> is passed in.

The README still says that the filesystem won't be searched; which is true for .swift-format, but no longer true for .swift-format-ignore.

The implication of that statement is that the filesystem won't be searched for the configuration (i.e., .swift-format file) specifically. We don't need it to apply to the .swift-format-ignore file.

If we want a way to ignore the ignore file, we could add something like a --force option to the command line interface, but I don't think we need it today.

README.md

samdeane · 2024-11-07T16:49:39Z

If we want a way to ignore the ignore file, we could add something like a --force option to the command line interface, but I don't think we need it today.

Yeah, I was thinking the same. I'll reword the docs.

allevato · 2024-11-07T20:08:28Z

Sources/SwiftFormat/Utilities/FileIterator.swift

+  repeat {
+    containingDirectory.deleteLastPathComponent()
+    let candidateFile = containingDirectory.appendingPathComponent(FileIterator.ignoreFileName)
+    if FileManager.default.isReadableFile(atPath: candidateFile.path) {


More than just checking for the presence of the file, I think we go the extra step of validating that it only contains * so that users aren't surprised later when we add more functionality and it subtly changes the behavior based on those contents. If it contains anything else, we should throw an error saying it's the only supported form (and turn that into a nice human-readable error message when we exit).

If we want to upgrade to gitignore like sytanx in the future, should we make the default value **/*?

AFAIK, * is sufficient because according to the documentation:

"An asterisk '*' matches anything except a slash."

"If there is a separator at the beginning or middle (or both) of the pattern, [...]. Otherwise the pattern may also match at any level below the .gitignore level."

"If there is a separator at the end of the pattern then the pattern will only match directories, otherwise the pattern can match both files and directories."

Sorry, got distracted, but am looking at this now.

One complication is that FileIterator doesn't throw, and instead relies on the caller to generate diagnostics for non-existent paths.

In order to report invalid ignore files inside FileIterator we'd either need to be able to:

throw an error from it

be able to emit diagnostics directly from it

return the path of the invalid ignore file (or a special pseudo-path), as if they were to be processed, then spot that path in the front end and emit an "invalid ignore file" diagnostic

I've gone for the third option, but it feels like a bit of a hack.

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

Sources/SwiftFormat/Core/IgnoreFile.swift

Sources/SwiftFormat/Utilities/FileIterator.swift

Sources/swift-format/Frontend/Frontend.swift

samdeane · 2024-12-11T13:56:45Z

Actually I've realised that the current design won't work as written, because when FileIterator encounters a directory, it collapses all subdirectories below it, and therefore will not check for the presence of nested .swift-format-ignore files.

FileIterator will need to be rewritten to check for an ignore file when entering a directory, and to pass that instance down to any subdirectories recursively.

samdeane · 2024-12-11T14:01:12Z

I'm happy to do this, but it raises a couple of design questions:

If we flow through the directory structure, inheriting the ignore settings from the parent directory, how should we treat sym links to other directories? Should they inherit their settings from their actual parent, or their logical parent in the context of the traversal?
We we're traversing a file tree and looking for .swift-format-ignore files, I wonder if we should also be looking for .swift-format files, and producing a (URL, Config) tuple for each element. It would be a lot more efficient than search for a nested config whilst processing every file.

samdeane · 2024-12-11T14:05:31Z

Slightly annoyingly, my original naive implementation was simple and worked fine, since it wasn't the front end doing the work.

I don't think it would be unreasonable to go with that simpler approach as the first iteration, and then refine it subsequently -- perfect being the enemy of good and all that. That said, I'm happy to look at reworking FileIterator instead.

samdeane · 2024-12-11T14:08:27Z

In other news, the current main branch (ed01028, without my changes) will not build with my version of Swift:

swift-driver version: 1.115.1 Apple Swift version 6.0.3 (swiftlang-6.0.3.1.9 clang-1600.0.30.1) Target: arm64-apple-macosx15.0

/Users/sam/Developer/Projects/swift-format/Sources/SwiftFormat/Rules/UseShorthandTypeNames.swift:50:78: error: type 'TypeSyntax?' has no member 'type'
 48 |     switch node.name.text {
 49 |     case "Array":
 50 |       guard case .type(let typeArgument) = genericArgumentList.firstAndOnly?.argument else {
    |                                                                              `- error: type 'TypeSyntax?' has no member 'type'
 51 |         newNode = nil
 52 |         break

(and a bunch of other errors)

samdeane · 2024-12-11T14:24:17Z

fwiw git bisect tells me that the problematic commit is c3b0c9f

allevato · 2024-12-11T14:29:45Z

Doing this in the frontend is still the right approach; the API layer shouldn't care about .swift-format-ignore files just as it doesn't care about .swift-format configuration files. Implementing it in the API layer as an incremental step would be putting us into a worse state than not having it, design-wise.

If we flow through the directory structure, inheriting the ignore settings from the parent directory, how should we treat sym links to other directories? Should they inherit their settings from their actual parent, or their logical parent in the context of the traversal?

I think the answer has to be the actual (physical) parent. Otherwise you could end up with different ignore states for the same file depending on how you got to it, which wouldn't be desirable. This does make the implementation slightly more complicated, but I think it would be workable to have FileIterator maintain a cache of some sort to keep track of the ignore-specs found in each directory it visits. When you visit a directory you look in the cache to see if it or any of its parents have loaded their ignore-specs, and in most cases, this will be fast because you got to that directory from its immediate parent. In the case where you follow a symlink into a different directory tree, you might need to look up all the way toward the root to find the ignore-specs (or know that they don't exist), but that would be rarer.

We we're traversing a file tree and looking for .swift-format-ignore files, I wonder if we should also be looking for .swift-format files, and producing a (URL, Config) tuple for each element. It would be a lot more efficient than search for a nested config whilst processing every file.

This sounds like a nice optimization! We flatten the iterator into an array in the frontend to do parallel processing so we don't want to have the full Configuration stored with each file URL (it would be identical across all files in the majority of cases), but returning a pair with the URL of the file to format and the URL of the relevant configuration would still be an improvement, I think. Then we can still leverage the existing configuration cache to load them on-demand, without the search each time.

The same caveat about which parent to use applies here as above. For symlinks, we'd want the configuration we choose to be the one from the physical parent, not the logical parent, because we don't want files to be formatted differently based on how you got there.

In other news, the current main branch (ed01028, without my changes) will not build with my version of Swift:

Is your swift-syntax checkout out of date? The change you referenced later was made to support an API change merged a few days ago in swift-syntax.

samdeane · 2024-12-11T16:38:25Z

Is your swift-syntax checkout out of date? The change you referenced later was made to support an API change merged a few days ago in swift-syntax.

Hmm, I don't have SWIFTCI_USE_LOCAL_DEPS set, so it should just be using what the package specifies. Unfortunately that is just the main branch, which I think is the problem.

Some combination of swift package purge-cache and swift package update convinced SwiftPM to pull the latest swift-syntax, which then worked with swift-format's main; unfortunately switching back to an old branch of swift-format then broke for similar reasons...

If it can't be pinned to a version, wouldn't it be better to pin it to an actual commit?

allevato · 2024-12-11T16:44:23Z

The main branches of swift-format and swift-syntax are always kept in sync to build against each other, so pinning swift-format's main branch to a specific commit wouldn't make sense. If you need to switch back to an older branch, rebasing it on top of main should be enough to get you in a working state again.

ahoppen

Sorry for the delay since my last review and thanks a lot for continuing to push this forward. I like the overall design and like the new tests that create the test files in a temporary directory. I left a few comments inline.

ahoppen · 2025-05-05T10:16:57Z

Sources/SwiftFormat/Core/IgnoreFile.swift

+      return nil
+    }
+
+    var containingDirectory = url.absoluteURL.standardized


Do we need to standardize the URL here? Say we have a directory structure like

/ LibA LibA.swift LibB Externals .swift-format-ignore = "*" LibA -> /LibA

Then, when running swift-format on /LibB/Externals/LibA/LibA.swift, I would expect formatting to be skipped because of /LibB/Externals/.swift-format-ignore. But with the current logic, we would standardize /LibB/Externals/LibA/LibA.swift to /LibA/LibA.swift in here and thus not find the .swift-format-ignore file.

ahoppen · 2025-05-05T10:19:03Z

Sources/SwiftFormat/Core/IgnoreFile.swift

+      if FileManager.default.isReadableFile(atPath: url.path) {
+        try self.init(contentsOf: url)
+        return
+      }


We should be able to skip this file system access and just check if self.init(contentsOf:) returns a non-nil value. Just trying to avoid race conditions where the file exists here but is gone by the time that self.init(contentsOf:) tries to read it.

Suggested change

if FileManager.default.isReadableFile(atPath: url.path) {

try self.init(contentsOf: url)

return

}

if let ignoreFile = try self.init(contentsOf: url) {

self = ignoreFile

return

}

ahoppen · 2025-05-05T10:20:44Z

Sources/SwiftFormat/Utilities/FileIterator.swift

+          } catch IgnoreFile.Error.invalidFile(let url, _) {
+            // we hit an invalid ignore file
+            // we return the path of the ignore file so that we can report an error
+            // and process the directory as normal
+            output = url
+          } catch {
+            // we hit another unexpected error; process the directory as normal
+          }


Why do we need to set output = url in the IgnoreFile.Error.invalidFile case but not in the generic error case?

ahoppen · 2025-05-05T10:22:07Z

Sources/SwiftFormat/Utilities/FileIterator.swift

+  return ignoreFile?.shouldProcess(url) ?? true
+}
+
+fileprivate extension URL {


Could we use isRoot from URL+isRoot.swift instead of defining a new implementation of it here?

ahoppen · 2025-05-05T10:23:39Z

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

@@ -0,0 +1,126 @@
+@_spi(Internal) import SwiftFormat


Could you add a copyright header to this test file?

ahoppen · 2025-05-05T10:30:51Z

Tests/SwiftFormatTests/Resources/Ignore Files/nested/.swift-format-ignore

Is this file still needed?

ahoppen · 2025-05-05T10:31:10Z

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

+  override func tearDown() {
+    // Clean up any test tree after each test.
+    if let testTreeURL {
+      // try? FileManager.default.removeItem(at: testTreeURL)
+    }
+  }


No longer needed?

ahoppen · 2025-05-05T10:36:25Z

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

+  func makeTempTree(_ tree: TestTree) throws -> URL {
+    if let testTreeURL {
+      try? FileManager.default.removeItem(at: testTreeURL)
+    }
+    let tempDir = FileManager.default.temporaryDirectory
+    let tempURL = tempDir.appendingPathComponent(UUID().uuidString)
+    try FileManager.default.createDirectory(at: tempURL, withIntermediateDirectories: true)
+    try writeTree(tree, to: tempURL)
+    testTreeURL = tempURL
+    return tempURL
+  }


What do you think about making this a withTempTree function that deletes the temporary directory after it executes the tests? Otherwise these trees will be left as artifacts. Ie something like

Suggested change

func makeTempTree(_ tree: TestTree) throws -> URL {

if let testTreeURL {

try? FileManager.default.removeItem(at: testTreeURL)

}

let tempDir = FileManager.default.temporaryDirectory

let tempURL = tempDir.appendingPathComponent(UUID().uuidString)

try FileManager.default.createDirectory(at: tempURL, withIntermediateDirectories: true)

try writeTree(tree, to: tempURL)

testTreeURL = tempURL

return tempURL

}

private func withTempTree(_ tree: TestTree, _ body: (URL) throws -> Void) throws {

let tempURL = FileManager.default.temporaryDirectory.appendingPathComponent("swift-format-test-scratch").appendingPathComponent(UUID().uuidString)

defer {

try? FileManager.default.removeItem(at: tempURL)

}

try FileManager.default.createDirectory(at: tempURL, withIntermediateDirectories: true) // I would move this to `writeTree`

try writeTree(tree, to: tempURL)

body(tempURL)

}

ahoppen · 2025-05-05T10:36:40Z

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

+  func writeTree(_ tree: TestTree, to root: URL) throws {
+    switch tree {
+    case let .file(name, contents):
+      print("Writing file \(name) to \(root)")


Leftover debug output?

ahoppen · 2025-05-05T10:37:29Z

Tests/SwiftFormatTests/Core/IgnoreFileTests.swift

+  }
+
+  /// Write a file or directory tree to the given root URL.
+  func writeTree(_ tree: TestTree, to root: URL) throws {


I would make this private

Suggested change

func writeTree(_ tree: TestTree, to root: URL) throws {

private func writeTree(_ tree: TestTree, to root: URL) throws {

samdeane mentioned this pull request Nov 7, 2024

Ability To Turn Off Formatting For Subdirectory #870

Open

samdeane marked this pull request as ready for review November 7, 2024 11:15

samdeane requested review from ahoppen, allevato, bnbarham and shawnhyam as code owners November 7, 2024 11:15

samdeane marked this pull request as draft November 7, 2024 11:24

samdeane marked this pull request as ready for review November 7, 2024 11:39

allevato reviewed Nov 7, 2024

View reviewed changes

samdeane force-pushed the main branch from f637617 to 549fafe Compare November 7, 2024 14:40

samdeane force-pushed the main branch 2 times, most recently from 9851d7a to bf42ce7 Compare November 7, 2024 15:15

allevato reviewed Nov 7, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

allevato reviewed Nov 7, 2024

View reviewed changes

samdeane requested review from allevato and andre-richter November 27, 2024 12:00

ahoppen reviewed Dec 3, 2024

View reviewed changes

samdeane force-pushed the main branch from 5485779 to 419a6db Compare December 11, 2024 12:53

samdeane added 5 commits December 11, 2024 12:53

Initial stab at option to disable all formatting.

eb29cb8

Updated config documentation in README.

356b55f

Fixed typo, initialiser.

021d6b3

Documented allDisabled key

3e4c5d2

Emit unmodified source if allDisabled is true.

8bade83

samdeane added 3 commits December 11, 2024 12:54

Filter urls to remove ignored ones.

fb8b06f

Tweaked readme.

5d85e22

Removed obsolete addition.

9ab91a5

samdeane force-pushed the main branch from 419a6db to 0178c63 Compare December 11, 2024 12:59

samdeane added 10 commits December 11, 2024 13:02

Oops - skipping initial directory...

7b35513

Documentation tweaks.

b2975de

Add minimal IgnoreFile abstraction.

ed1d3c4

Comment for IgnoreFile class

3bc036e

Added IgnoreFile unit tests

cff2b22

Added test for nested directory.

e9861ca

Added string constructor. Don't skip dir.

ba343ca

Removed test resources

c9a14e3

fixed nested test with explanatory comment

707061f

Fixed merge conflict

85840d6

samdeane force-pushed the main branch from 0178c63 to 85840d6 Compare December 11, 2024 13:08

Actually, this test should fail (currently)

5b5db57

MahdiBM mentioned this pull request Dec 14, 2024

Use Apple's GitHub Workflows linting script vapor/ci#42

Merged

allevato mentioned this pull request Feb 14, 2025

FR: .swiftformatignore, etc to ignore Pods/ and other folders when recursively formatting #931

Closed

MahdiBM mentioned this pull request Feb 15, 2025

Make generated files // swift-format-ignore apple/swift-openapi-generator#738

Open

This was referenced Feb 19, 2025

Add Swift formatting locally and in CI DefinedNet/mobile_nebula#256

Closed

Use swift-format for Swift file formatting DefinedNet/mobile_nebula#261

Merged

ahoppen reviewed May 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ability To Turn Off Formatting For Subdirectory #873

Ability To Turn Off Formatting For Subdirectory #873

samdeane commented Nov 7, 2024 •

edited

Loading

allevato left a comment

samdeane commented Nov 7, 2024

samdeane commented Nov 7, 2024

samdeane commented Nov 7, 2024

allevato commented Nov 7, 2024

samdeane commented Nov 7, 2024

allevato commented Nov 7, 2024

samdeane commented Nov 7, 2024

allevato Nov 7, 2024

andre-richter Nov 7, 2024

allevato Nov 7, 2024

samdeane Nov 27, 2024 •

edited

Loading

samdeane commented Dec 11, 2024

samdeane commented Dec 11, 2024 •

edited

Loading

samdeane commented Dec 11, 2024

samdeane commented Dec 11, 2024 •

edited

Loading

samdeane commented Dec 11, 2024

allevato commented Dec 11, 2024

samdeane commented Dec 11, 2024

allevato commented Dec 11, 2024

ahoppen left a comment

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

ahoppen May 5, 2025

	func writeTree(_ tree: TestTree, to root: URL) throws {
	private func writeTree(_ tree: TestTree, to root: URL) throws {

Ability To Turn Off Formatting For Subdirectory #873

Are you sure you want to change the base?

Ability To Turn Off Formatting For Subdirectory #873

Conversation

samdeane commented Nov 7, 2024 • edited Loading

Motivation

allevato left a comment

Choose a reason for hiding this comment

samdeane commented Nov 7, 2024

samdeane commented Nov 7, 2024

samdeane commented Nov 7, 2024

allevato commented Nov 7, 2024

samdeane commented Nov 7, 2024

allevato commented Nov 7, 2024

samdeane commented Nov 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samdeane Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

samdeane commented Dec 11, 2024

samdeane commented Dec 11, 2024 • edited Loading

samdeane commented Dec 11, 2024

samdeane commented Dec 11, 2024 • edited Loading

samdeane commented Dec 11, 2024

allevato commented Dec 11, 2024

samdeane commented Dec 11, 2024

allevato commented Dec 11, 2024

ahoppen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samdeane commented Nov 7, 2024 •

edited

Loading

samdeane Nov 27, 2024 •

edited

Loading

samdeane commented Dec 11, 2024 •

edited

Loading

samdeane commented Dec 11, 2024 •

edited

Loading