Skip to content

Commit 5a74b0f

Browse files
authored
fix(local-dev): drop sbt build knobs that ship an unrunnable dist (apache#6097)
### What changes were proposed in this PR? `build_all` in `bin/local-dev/main.sh` passed three CLI-only sbt knobs that each made `sbt <svc>/dist` produce a dist that packaged and reported success but could not run. This PR removes all three and keeps only the heap/GC flags. - `-Dsbt.pipelining=true`: inter-project dependencies are consumed as early-output signature jars, so the dist drops the real `dao`/`auth`/`config`/`resource` jars from `lib/` and every service dies at startup with `NoClassDefFoundError` on a sibling-module class (e.g. `org/apache/texera/dao/SqlServer$`). - `set every (Compile / doc / sources) := Seq.empty`: `set every` ignores the written scope and zeroes the shared `sources` key in every scope, including `Compile / sources`, so nothing compiles, no main class is discovered, and no `bin/<service>` launcher is generated. - `set every (Compile / packageDoc / publishArtifact) := false`: the same `set every` trap turns off `publishArtifact` for the main jar artifacts too, so the inter-project dependency jars drop out of the dist. The regression tests in `test_local_dev_sh.sh` are flipped to guards that fail if any of these knobs is reintroduced. Skipping scaladoc has to be done per-project in `build.sbt`, not with a global `set every`, so it is left out until it can be done safely. ### Any related issues, documentation, discussions? Closes apache#6096 ### How was this PR tested? Verified a clean multi-project dist bundles the `dao`/`auth`/`config`/`resource` jars and a working `bin/<service>` launcher, and that `bin/local-dev/tests/test_local_dev_sh.sh` passes. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8)
1 parent bc5accf commit 5a74b0f

2 files changed

Lines changed: 36 additions & 14 deletions

File tree

bin/local-dev/main.sh

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1916,15 +1916,28 @@ build_all() {
19161916
fi
19171917
19181918
# CLI-only build knobs applied to the local-dev entrypoint (build.sbt
1919-
# untouched): skip scaladoc (biggest single win on dist), pipeline
1920-
# signature-then-body compile across projects, raise heap + G1GC.
1919+
# untouched): raise heap + G1GC. Nothing here may alter what the dist
1920+
# actually ships.
1921+
#
1922+
# The removed knobs each produced a dist that packaged and reported success
1923+
# but could not run:
1924+
# * -Dsbt.pipelining=true — inter-project deps are consumed as
1925+
# early-output signature jars and the dist drops the real dependency
1926+
# jars (dao/auth/config/resource) from lib/, so every service dies at
1927+
# startup with NoClassDefFoundError on a sibling-module class.
1928+
# * set every (Compile / doc / sources) := Seq.empty — `set every`
1929+
# ignores the written scope and zeroes the shared `sources` key in
1930+
# EVERY scope (incl. Compile / sources): nothing compiles, no main
1931+
# class is discovered, and no bin/<service> launcher is generated.
1932+
# * set every (Compile / packageDoc / publishArtifact) := false — same
1933+
# `set every` trap: it turns off `publishArtifact` for the main jar
1934+
# artifacts too, so inter-project dependency jars drop out of the dist.
1935+
# Skipping scaladoc has to be done per-project in build.sbt, not with a
1936+
# global `set every`, so it is left out here until done safely.
19211937
local -a sbt_opts=(
19221938
-no-colors
19231939
-J-Xmx4g
19241940
-J-XX:+UseG1GC
1925-
-Dsbt.pipelining=true
1926-
'set every (Compile / doc / sources) := Seq.empty'
1927-
'set every (Compile / packageDoc / publishArtifact) := false'
19281941
)
19291942
19301943
if [[ "${FRESH:-false}" == "true" ]]; then

bin/local-dev/tests/test_local_dev_sh.sh

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -413,20 +413,29 @@ for fn in cmd_up cmd_auto; do
413413
fi
414414
done
415415

416-
# 24) build_all applies CLI-only sbt speedups: skip scaladoc + enable
417-
# pipelining. Keeping the knobs here (not in build.sbt) means CI and
418-
# standalone sbt invocations are unaffected.
416+
# 24) build_all's CLI-only sbt knobs must never alter what the dist ships.
417+
# Three knobs are banned because each produced a dist that packaged fine
418+
# but could not run: `-Dsbt.pipelining=true` and the two `set every`
419+
# settings all drop inter-project dependency jars or the bin/<service>
420+
# launcher from the dist. Match against code only (comments name the
421+
# banned knobs to explain them).
419422
build_body=$(awk '/^build_all\(\)/{f=1} f{print} f&&/^}/{exit}' \
420423
"$REPO_ROOT/bin/local-dev/main.sh")
421-
if [[ "$build_body" == *"-Dsbt.pipelining=true"* ]]; then
422-
_pass "build_all: sbt invocation enables -Dsbt.pipelining=true"
424+
build_code=$(printf '%s\n' "$build_body" | grep -vE '^[[:space:]]*#')
425+
if [[ "$build_code" == *"-Dsbt.pipelining=true"* ]]; then
426+
_fail "build_all: '-Dsbt.pipelining=true' drops inter-project dependency jars from the dist"
423427
else
424-
_fail "build_all: -Dsbt.pipelining=true flag missing"
428+
_pass "build_all: does not enable sbt pipelining"
425429
fi
426-
if [[ "$build_body" == *"Compile / doc / sources) := Seq.empty"* ]]; then
427-
_pass "build_all: scaladoc sources emptied via 'set every'"
430+
if [[ "$build_code" == *"doc / sources) := Seq.empty"* ]]; then
431+
_fail "build_all: 'set every (Compile / doc / sources) := Seq.empty' empties Compile/sources → no launcher"
428432
else
429-
_fail "build_all: doc-skip 'set every' setting missing"
433+
_pass "build_all: does not clobber Compile/sources via 'set every ... doc / sources'"
434+
fi
435+
if [[ "$build_code" == *"packageDoc / publishArtifact) := false"* ]]; then
436+
_fail "build_all: 'set every (Compile / packageDoc / publishArtifact) := false' drops dependency jars from the dist"
437+
else
438+
_pass "build_all: does not disable publishArtifact via 'set every ... packageDoc'"
430439
fi
431440

432441
# 25) --skip=<svc> flows into the sbt build: skipped JVM services drop out

0 commit comments

Comments
 (0)