Skip to content

fix: create parent directories for nested clonePath#1652

Merged
dkwon17 merged 1 commit into
devfile:mainfrom
snecklifter:fix/create-clone-path-parent-dirs
Jun 22, 2026
Merged

fix: create parent directories for nested clonePath#1652
dkwon17 merged 1 commit into
devfile:mainfrom
snecklifter:fix/create-clone-path-parent-dirs

Conversation

@snecklifter

@snecklifter snecklifter commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • When a devfile project specifies a nested clonePath (e.g. back/devfile-rest), os.Rename fails because the intermediary parent directories don't exist
  • Adds os.MkdirAll(path.Dir(...)) calls before os.Rename in both the git and zip project setup paths to ensure parent directories are created
  • For flat clonePath values, path.Dir resolves to the already-existing root so MkdirAll is a no-op

Fixes: eclipse-che/che#23877

Test plan

  • Start a workspace with a devfile containing multiple projects with nested clonePath values (e.g. back/devfile-rest, ui/devfile-ui)
  • Verify all projects are cloned into the correct subdirectory structure under /projects
  • Verify single-level clonePath values still work as before

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Projects can now be successfully cloned into deeply nested destination paths without requiring manual parent directory creation, improving workflow flexibility for complex project structures.
  • Tests

    • Added comprehensive test coverage for nested clone path scenarios, verifying both git and zip-based project cloning into multiple directory levels.

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown

Hi @snecklifter. Thanks for your PR.

I'm waiting for a devfile member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 711a1434-30e0-40c9-9758-0eb3caaefcbe

📥 Commits

Reviewing files that changed from the base of the PR and between cdc8050 and 437833d.

📒 Files selected for processing (3)
  • project-clone/internal/git/setup.go
  • project-clone/internal/zip/setup.go
  • project-clone/test/nested-clone-path-test.devworkspace.yaml

📝 Walkthrough

Walkthrough

Adds os.MkdirAll calls in the git (doInitialGitClone, copyProjectFromTmpDir) and zip (SetupZipProject) project-clone setup paths to create intermediate parent directories before temporary clone paths and final destination paths are used. A new DevWorkspace YAML integration test validates nested clonePath scenarios.

Changes

Parent Directory Creation Fix with Integration Test

Layer / File(s) Summary
MkdirAll in git and zip setup before clone and rename
project-clone/internal/git/setup.go, project-clone/internal/zip/setup.go
doInitialGitClone creates the parent of tmpClonePath via os.MkdirAll before cloning. copyProjectFromTmpDir creates the parent of projectPath before any rename, and the subdir branch is updated to use the precomputed projectPath with an added log line. SetupZipProject adds os.MkdirAll for both the temporary extraction path and the final projectPath before renaming, wrapping errors on failure.
Integration test: nested clonePath DevWorkspace manifest
project-clone/test/nested-clone-path-test.devworkspace.yaml
New ephemeral DevWorkspace manifest clones four git projects into nested (back/api, back/worker, ui/app) and flat (flat-project) clonePath locations. An inline bash script validates directory existence, default_branch_name checkout, and origin remote URL for each cloned project.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 Hop hop, the path must be there,
Before we clone into the air!
MkdirAll clears the way,
Nested folders here to stay.
No more errors, hip hooray! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding directory creation for nested clonePath specifications.
Linked Issues check ✅ Passed The PR fully addresses the linked issue #23877 by adding os.MkdirAll calls in git and zip setup code to create parent directories before rename operations.
Out of Scope Changes check ✅ Passed All changes are directly related to fixing the nested clonePath issue: two setup files implement the fix and one test file validates it.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@dkwon17

dkwon17 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Thank you @snecklifter LGTM, could you please add some tests in https://github.com/devfile/devworkspace-operator/tree/main/project-clone/test?

When a devfile project specifies a nested clonePath (e.g.
"back/devfile-rest"), the intermediary directories are never created,
causing os.Rename to fail when moving the cloned project to its final
destination.

Add os.MkdirAll calls before os.Rename in both the git and zip
project setup paths to ensure parent directories exist. Add an
integration test fixture exercising nested clonePath with multiple
projects sharing intermediary directories.

Fixes: eclipse-che/che#23877

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Chris Brown <chribrow@redhat.com>
@snecklifter snecklifter force-pushed the fix/create-clone-path-parent-dirs branch from 847a41d to 437833d Compare June 19, 2026 21:36
@snecklifter

Copy link
Copy Markdown
Contributor Author

Thanks @dkwon17 — I've added an integration test fixture at project-clone/test/nested-clone-path-test.devworkspace.yaml that covers nested clonePath with multiple projects sharing intermediary directories, plus a flat project to ensure no regression. Squashed into a single commit.

@rohanKanojia

Copy link
Copy Markdown
Member

Tested this PR and it seems to work as expected

  • Created this devworkspace (it verifies cloned dir structure):
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: nested-clone-path-test
  labels:
    app.kubernetes.io/name: devworkspace-project-clone-tests
    app.kubernetes.io/part-of: devworkspace-operator
  annotations:
    controller.devfile.io/debug-start: "true"
spec:
  started: true
  routingClass: 'basic'
  template:
    attributes:
      controller.devfile.io/storage-type: ephemeral
    variables:
      test_runner_image: quay.io/devfile/project-clone:next # Requires git, bash
      main_repo: https://github.com/devfile/devworkspace-operator.git
      default_branch_name: main
    projects:
      - name: backend-api
        clonePath: back/api
        git:
          remotes:
            origin: "{{main_repo}}"
      - name: backend-worker
        clonePath: back/worker
        git:
          remotes:
            origin: "{{main_repo}}"
      - name: frontend-app
        clonePath: ui/app
        git:
          remotes:
            origin: "{{main_repo}}"
      - name: flat-project
        git:
          remotes:
            origin: "{{main_repo}}"
    components:
      - name: test-project-clone
        container:
          image: "{{test_runner_image}}"
          memoryLimit: 512Mi
          mountSources: true
          command:
            - "/bin/bash"
            - "-c"
            - |
              set -e

              fail() {
                echo "[ERROR] $1"
                echo "[ERROR] See project-clone logs: "
                echo "[ERROR]    oc logs -n $DEVWORKSPACE_NAMESPACE deploy/$DEVWORKSPACE_ID -c project-clone"
                exit 1
              }

              if [ -f "${PROJECTS_ROOT}/project-clone-errors.log" ]; then
                echo "==== BEGIN PROJECT CLONE LOGS ===="
                sed 's/^/    /g' "${PROJECTS_ROOT}/project-clone-errors.log"
                echo "====  END PROJECT CLONE LOGS  ===="
                echo -e "\n\n"
              fi

              for project_dir in "back/api" "back/worker" "ui/app" "flat-project"; do
                if [ ! -d "${PROJECTS_ROOT}/${project_dir}" ]; then
                  fail "Project at ${project_dir} not cloned successfully"
                fi
              done

              echo "Testing nested clonePath project: back/api"
              cd "${PROJECTS_ROOT}/back/api"
              branch_name=$(git rev-parse --abbrev-ref HEAD)
              if [ "$branch_name" != "{{default_branch_name}}" ]; then
                fail "Project back/api does not have default branch checked out"
              fi
              remote_url=$(git config remote.origin.url)
              if [ "$remote_url" != "{{main_repo}}" ]; then
                fail "Remote 'origin' not configured for back/api"
              fi
              echo "back/api: on $branch_name, remotes configured"

              echo "Testing nested clonePath project: back/worker"
              cd "${PROJECTS_ROOT}/back/worker"
              branch_name=$(git rev-parse --abbrev-ref HEAD)
              if [ "$branch_name" != "{{default_branch_name}}" ]; then
                fail "Project back/worker does not have default branch checked out"
              fi
              remote_url=$(git config remote.origin.url)
              if [ "$remote_url" != "{{main_repo}}" ]; then
                fail "Remote 'origin' not configured for back/worker"
              fi
              echo "back/worker: on $branch_name, remotes configured"

              echo "Testing nested clonePath project: ui/app"
              cd "${PROJECTS_ROOT}/ui/app"
              branch_name=$(git rev-parse --abbrev-ref HEAD)
              if [ "$branch_name" != "{{default_branch_name}}" ]; then
                fail "Project ui/app does not have default branch checked out"
              fi
              remote_url=$(git config remote.origin.url)
              if [ "$remote_url" != "{{main_repo}}" ]; then
                fail "Remote 'origin' not configured for ui/app"
              fi
              echo "ui/app: on $branch_name, remotes configured"

              echo "Testing flat clonePath project (no nesting)"
              cd "${PROJECTS_ROOT}/flat-project"
              branch_name=$(git rev-parse --abbrev-ref HEAD)
              if [ "$branch_name" != "{{default_branch_name}}" ]; then
                fail "Project flat-project does not have default branch checked out"
              fi
              remote_url=$(git config remote.origin.url)
              if [ "$remote_url" != "{{main_repo}}" ]; then
                fail "Remote 'origin' not configured for flat-project"
              fi
              echo "flat-project: on $branch_name, remotes configured"

              echo "[SUCCESS] Test succeeded. Sleeping indefinitely"
              tail -f /dev/null
  • Once devworkspace ready observed logs:
🔹 Logs for container: project-clone
2026/06/22 11:03:46 Using temporary directory /projects/project-clone-4070239715
2026/06/22 11:03:46 Read DevWorkspace at /devworkspace-metadata/flattened.devworkspace.yaml
2026/06/22 11:03:46 Processing project backend-api
2026/06/22 11:03:46 Cloning project backend-api to /projects/project-clone-4070239715/back/api
Cloning into '/projects/project-clone-4070239715/back/api'...
2026/06/22 11:03:53 Cloned project backend-api to /projects/project-clone-4070239715/back/api
2026/06/22 11:03:53 Setting up remotes for project backend-api
2026/06/22 11:03:53 Fetched remote origin at https://github.com/devfile/devworkspace-operator.git
2026/06/22 11:03:53 Moving cloned project backend-api from temporary directory /projects/project-clone-4070239715/back/api to /projects/back/api
2026/06/22 11:03:53 Processing project backend-worker
2026/06/22 11:03:53 Cloning project backend-worker to /projects/project-clone-4070239715/back/worker
Cloning into '/projects/project-clone-4070239715/back/worker'...
2026/06/22 11:04:00 Cloned project backend-worker to /projects/project-clone-4070239715/back/worker
2026/06/22 11:04:00 Setting up remotes for project backend-worker
2026/06/22 11:04:02 Fetched remote origin at https://github.com/devfile/devworkspace-operator.git
2026/06/22 11:04:02 Moving cloned project backend-worker from temporary directory /projects/project-clone-4070239715/back/worker to /projects/back/worker
2026/06/22 11:04:02 Processing project frontend-app
2026/06/22 11:04:02 Cloning project frontend-app to /projects/project-clone-4070239715/ui/app
Cloning into '/projects/project-clone-4070239715/ui/app'...
2026/06/22 11:04:10 Cloned project frontend-app to /projects/project-clone-4070239715/ui/app
2026/06/22 11:04:10 Setting up remotes for project frontend-app
2026/06/22 11:04:11 Fetched remote origin at https://github.com/devfile/devworkspace-operator.git
2026/06/22 11:04:11 Moving cloned project frontend-app from temporary directory /projects/project-clone-4070239715/ui/app to /projects/ui/app
2026/06/22 11:04:11 Processing project flat-project
2026/06/22 11:04:11 Cloning project flat-project to /projects/project-clone-4070239715/flat-project
Cloning into '/projects/project-clone-4070239715/flat-project'...
2026/06/22 11:04:19 Cloned project flat-project to /projects/project-clone-4070239715/flat-project
2026/06/22 11:04:19 Setting up remotes for project flat-project
2026/06/22 11:04:19 Fetched remote origin at https://github.com/devfile/devworkspace-operator.git
2026/06/22 11:04:19 Moving cloned project flat-project from temporary directory /projects/project-clone-4070239715/flat-project to /projects/flat-project

🔹 Logs for container: test-project-clone
Testing nested clonePath project: back/api
back/api: on main, remotes configured
Testing nested clonePath project: back/worker
back/worker: on main, remotes configured
Testing nested clonePath project: ui/app
ui/app: on main, remotes configured
Testing flat clonePath project (no nesting)
flat-project: on main, remotes configured
[SUCCESS] Test succeeded. Sleeping indefinitely

@openshift-ci

openshift-ci Bot commented Jun 22, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rohanKanojia, snecklifter
Once this PR has been reviewed and has the lgtm label, please assign dkwon17 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dkwon17

dkwon17 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

@snecklifter thank you! Merging.

@dkwon17 dkwon17 merged commit 3bbd19a into devfile:main Jun 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DevFile] When clone path is set, intermediary non existing folder is not created

3 participants