- notifications - lists.xwiki.org

[xwiki/xwiki-docker] 187b5c: XDOCKER-18: Automate updating the stable version (...
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/master Home: https://github.com/xwiki/xwiki-docker Commit: 187b5c3569a5f002f049cae459e150d90a83094a https://github.com/xwiki/xwiki-docker/commit/187b5c3569a5f002f049cae459e150… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M .github/workflows/docker-build.yml M .github/workflows/gradlew-check.yml M 16/mariadb-tomcat/docker-compose.yml M 16/mysql-tomcat/docker-compose.yml M 17/mariadb-tomcat/docker-compose.yml M 17/mysql-tomcat/docker-compose.yml M 18.4/mariadb-tomcat/docker-compose.yml M 18.4/mysql-tomcat/docker-compose.yml M 18/mariadb-tomcat/docker-compose.yml M 18/mysql-tomcat/docker-compose.yml M README.md M build.gradle A gradle/helpers.gradle A gradle/official-image.gradle A gradle/release.gradle A gradle/run-instances.gradle A gradle/update-versions.gradle M template/docker-compose.yml A versions.json Log Message: ----------- XDOCKER-18: Automate updating the stable version (#92) * XDOCKER-18: Automate updating the stable version Automate the manual "Update Docker Images" release step as Gradle tasks, mirroring the existing updateLibreOffice precedent: * updateXWiki: resolve the latest final XWiki version on each existing cycle's line from XWiki's Maven metadata and, for the changed cycles, download the WAR and compute its sha256. * updateJDBC: read the three JDBC driver versions from each cycle's own xwiki-platform POM and, for the changed ones, download the driver from Maven Central and compute its sha256. Driver versions/sha256 are now stored per cycle, replacing the shared mysql/mariadb/postgres maps. * smokeTest: boot the postgres-tomcat variant of each changed cycle with docker compose, wait until its REST API reports the expected XWiki version, then tear it down. * runAll/stopAll: start each selected cycle/variant on its own host port as an isolated compose project and leave them running for manual inspection. * release: the pre-push umbrella chaining updateXWiki, updateJDBC, updateLibreOffice, generate, generateWorkflows and smokeTest. * submitOfficialImage: generate the docker-library 'library/xwiki' file from the version data model and open the official-images Pull Request from the maintainer's fork (-PdryRun previews the file and the diff only). The version data model (per-cycle XWiki + JDBC driver versions/sha256, per-cycle role and base image, and the shared LibreOffice values) moves out of build.gradle into the versions.json data file, keeping the configuration separate from the build logic and letting the update tasks persist it with JsonOutput. The values it starts with are the ones master currently generates from, so this change regenerates no image. The build logic itself is split out of build.gradle into scripts applied from gradle/: helpers.gradle (HTTP, hashing and process helpers), update-versions.gradle, run-instances.gradle, official-image.gradle and release.gradle. They exchange the model and the helpers through the project's extra properties and bind them at configuration time, so that no task action reaches for the project while it runs (deprecated in Gradle 9, removed in Gradle 10). External commands go through Groovy's Process API since Project.exec is gone from Gradle 9. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Never downgrade a JDBC driver below the pinned version The JDBC driver version declared in a cycle's own xwiki-platform POM is a floor, not an exact target, so updateJDBC now keeps whatever is pinned in versions.json when the POM declares something older. A pin ahead of the POM is a deliberate bump made after that cycle was released (a security fix, or a Renovate upgrade such as XWIKI-24657 for the MySQL connector), and applying the POM value literally would silently revert it and rebuild the images for nothing. Upgrades still apply, including the ones crossing a driver major, but those now say so in the log since the release manager reviews the diff before committing it. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Fix the runAll/stopAll variant selection being shadowed * The 'ext.variants' extra property holding the DB variants collided with the '-Pvariants' project property that 'runAll'/'stopAll' read to narrow their selection down. Since the extra property is set before the script declaring those tasks is applied, 'project.hasProperty("variants")' was always true and 'project.property("variants")' always returned the variant List rather than the command line String. Both tasks thus failed with "No signature of method: java.util.ArrayList.tokenize()" on every invocation, with or without the flag. * Renamed the extra property to 'imageVariants' so that it no longer shares the '-Pvariants' name. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Build the image in smokeTest instead of pulling it * The generated docker-compose.yml has no 'build' section, only 'image: xwiki:<version>-<variant>', so 'smokeTest' pulled that tag from DockerHub. At release time that tag does not exist yet, since the official image for a version is only built once the 'submitOfficialImage' pull request has been merged, which happens after 'release' runs. The boot therefore failed with "manifest unknown" in exactly the scenario the task exists for, and when run on an already published version it silently exercised that published image rather than the freshly regenerated Dockerfile. * Build the image locally before booting it, so that the smoke test covers the Dockerfile that was just generated. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Fix the captured command output being truncated or empty * 'capture' consumed the process output into two writers and then merely waited for the process, which races with the threads draining its streams: the process can exit while they still have bytes to write, so the captured stdout came back short or altogether empty. Measured on a 4 MB output, 4 runs out of 50 returned an empty String. * Everything 'capture' reads either decides something or ends up in the content submitted upstream, so a silently short read is not an option: 'git status --porcelain' coming back empty passes the dirty tree guard of 'submitOfficialImage', and an empty 'git log' answer emits an empty 'GitCommit' into the official images file. * Use 'waitForProcessOutput', which waits for the process and for both draining threads (and still lets 'exitValue' be read afterwards). Same measurement: 0 mismatches out of 50. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Build the images in runAll instead of pulling them * 'runAll' rewrote the ports and the container names of the generated docker-compose.yml but left its 'image' line alone, so it pulled 'xwiki:<version>-<variant>' from DockerHub exactly like 'smokeTest' used to. At release time that tag does not exist yet, so the boot failed with "manifest unknown" precisely when the images worth looking at are the ones just regenerated, and on an already published version it silently ran the published image instead of them. * Extract the build step into a 'buildImage' helper shared with 'smokeTest' and build each instance before starting it. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Smoke test the cycles changed by any of the update tasks * 'smokeTest' picked the cycles to boot from what 'updateXWiki' reported, so a release bumping only a JDBC driver or the shared LibreOffice version regenerated images and then tested nothing at all, although those are the changes carrying a brand new download URL and sha256, i.e. the ones most likely to break the build. * 'updateJDBC' now reports the cycles whose drivers it changed, and 'updateLibreOffice' reports that all of them changed, since the LibreOffice version is shared by every cycle. The holder they share with 'smokeTest' is renamed to 'versionUpdate' accordingly. * Order 'smokeTest' after the update tasks themselves and not only after the 'generate' they order before, so that it still sees what they changed when it is chained without 'generate' (e.g. './gradlew updateJDBC smokeTest'). * Log the cycles being tested, since which ones they are is now the result of what three tasks found. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Report the docker output as it comes instead of swallowing it * The docker commands went through 'capture', which buffers the whole output and returns it, so booting an instance showed nothing at all until it was over. A 'docker build' downloading the XWiki WAR and the LibreOffice archive takes minutes and says what it is doing all along, and none of that was reported; the log was only held in memory to be thrown away, or quoted in an exception once the command had already failed. * Add a 'stream' helper next to 'capture' for the commands whose output is worth watching rather than parsing, and use it for 'docker build' and the 'docker compose' calls of 'smokeTest', 'runAll' and 'stopAll'. It merges stderr into stdout, reports each line as it arrives, and reads the stream to its end before waiting for the process. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Build the images with the classic builder like the CI does * 'smokeTest' and 'runAll' built with whatever builder the local Docker defaults to, i.e. BuildKit, while the Docker Official Images infrastructure uses the classic builder and the Docker Build workflow sets DOCKER_BUILDKIT=0 to match it. The two do not accept the same Dockerfiles: the classic builder leaves Docker's automatic platform arguments empty, for one, so a Dockerfile reading TARGETARCH builds fine locally and breaks where the image is really built. * Ask for the classic builder in 'buildImage' too, so that a green boot check says something about the build that will actually be run. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Require the commits submitted upstream to be on master * 'submitOfficialImage' refused an unpushed HEAD by looking for it in any remote branch at all, so running it from a Pull Request branch passed the check although docker-library fetches the images from the master branch of GitRepo alone. The GitCommit values emitted for such a run point at commits nobody but GitHub can fetch. * Check that HEAD is on the branch docker-library actually fetches, and fetch that remote-tracking branch first since it goes stale as soon as the change is pushed from elsewhere (a merged Pull Request, another clone). Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * List the tasks in './gradlew tasks' * None of the tasks declared a group, so they all landed in the ungrouped bucket that './gradlew tasks' only prints with '--all'. The plain listing showed none of the entry points the README documents, and no description of what they do or of the order they are meant to be run in. * Give each of them a group and a description: 'generation' for the two tasks generating the images and the workflow, 'release' for the version updates and the two release entry points, and 'verification' for the ones booting images. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Compare the versions without failing on a qualifier * 'compareVersions' parsed every dot-separated component as an int, so a version carrying anything but digits threw a NumberFormatException. Every version it is given is plain numbers today, but it decides whether a driver found in a POM is an upgrade over the pinned one, i.e. it reads values this repository does not control, and aborting a release over the day one of them gains a qualifier is a poor trade. * Compare each component on the number it starts with, falling back to comparing the components as text when those numbers are equal, so that a qualifier orders instead of failing. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Look the JDBC driver properties up by their exact name * The regex reading a driver version out of a cycle's POM interpolated the property name as it is, so the '.' of 'mysql.version' was left as the regex metacharacter matching any character. Nothing in the POMs happens to match those patterns loosely today, but a property whose name only resembles the one being read would be picked up. * Quote the property name so that it is matched literally. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Leave versions.json alone when LibreOffice has not moved * 'updateLibreOffice' rewrote the whole data file on every run, including when it had just resolved the very version and checksums already pinned, unlike the two other update tasks which only write what they changed. * Write only when the version or one of the two checksums differs, and report a checksum changing under an unchanged version for what it is, since that means the archive it was read from was republished. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Never close an official images Pull Request still open * 'submitOfficialImage' names its branch after the stable version and deleted that branch on the fork before recreating it. Running it twice for the same version, which happens whenever something has to be corrected, deleted the ref of the branch the pending Pull Request was opened from, and GitHub closes a Pull Request whose head branch is gone. A submission still being reviewed upstream is not ours to close. * Refuse to run when a Pull Request is still open from that branch, naming it so it can be looked at. * Also fail with an explanation rather than a NullPointerException when no cycle declares the 'stable' role, since that is the version the branch and the commit message are named after. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Say what binding the model at configuration time really buys * The comments explaining why the applied scripts bind the model and the helpers into local variables claimed that no task action reaches for the project while it runs, which is not what the code does: 'generate' and 'generateWorkflows' call 'copy', and the tasks booting images resolve their paths with 'file'. A comment promising more than the code delivers is worse than no comment, and this one would have the next reader believe the build is configuration cache compatible when it is not. * State what the binding actually avoids, namely reading the project properties from inside the task actions, and say where the build still stands with the configuration cache. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Give each booted instance a compose project of its own * 'smokeTest' ran 'docker compose up -d' and 'down -v' straight in the version/variant directory, so its compose project was the directory basename - 'postgres-tomcat', which every cycle shares and which is also what a developer gets from 'cd 18/postgres-tomcat && docker compose up'. Booting cycle 18 while such an instance was running failed on the container name and the port 8080 already taken, and the teardown in the 'finally' then removed that instance's containers and, with '-v', its data volumes. A release check has no business deleting someone's wiki. * Rename 'runAllCompose' to 'composeInstance', move it above 'smokeTest' and use it there too, so the smoke test gets the port and container renaming plus the '-p' project that 'runAll' already had. Both derive their project name through 'composeProject', which prefixes it with the task so tearing one down never reaches the other. * Publish the smoke test on 18080 rather than 8080, so that a port a developer is already using cannot fail a boot for a reason unrelated to the image being checked. * Depend on 'generate' in 'smokeTest' and 'runAll'. The image is tagged with the model's XWiki version while the compose file names the one in the generated '.env', so './gradlew updateXWiki smokeTest' used to boot the previously published image and then wait ten minutes for a version it could never report. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Retry a download that failed rather than abort the release * 'release' streams the XWiki WAR and the LibreOffice archives, hundreds of megabytes from hosts that fail transiently often enough that the generated Dockerfile retries its own downloads for that very reason (CURL_OPTIONS in template/Dockerfile). Here a single blip aborted the run, possibly after 'versions.json' had already been rewritten for the cycles handled before it, leaving the release manager to work out how far it got. * Fold the redirect-following loop that 'httpGetText' and 'sha256OfUrl' each carried into a single 'httpGet' that hands the response stream to a consumer, and give it three attempts with a growing delay. A failure while the consumer is reading counts too, since that is where a large download dies. * Only retry what can get better on its own: an IOException and the 408/429/5xx statuses. A 404 on a version that does not exist still fails at once instead of burning the delays first. * Convert the command to plain Strings in 'stream'. ProcessBuilder only turns its list into an array when the process starts, so a GString slipped into it failed there with an ArrayStoreException rather than where it was written - and 'capture', which the same scripts use interchangeably, takes GStrings in its stride. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Reject a cycle role that docker-library would not understand * The 'role' of a cycle is not only ours: it drives the tag aliases and the section comments of the official Docker library file that 'submitOfficialImage' submits to docker-library. A typo in 'versions.json' went through unnoticed, emitting a literal 'null' section header, silently dropping that cycle's stable-*/lts-* aliases and putting '(null)' in the Pull Request body - noticed, if at all, by a reviewer upstream. * Check the roles when build.gradle reads the model, and check that exactly one cycle is the stable one, since the branch and the commit message of that Pull Request are named after its version. * Drop the check 'submitOfficialImage' made for a missing stable cycle, now that it cannot happen, and say where the invariant comes from. * Also drop 'ext.versionsFile', which nothing read: 'writeVersions' closes over it. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Hash the WAR at the URL the image downloads it from * 'updateXWiki' resolved the versions and computed the sha256 from nexus.xwiki.org while the generated Dockerfile downloads the WAR from maven.xwiki.org, on the argument that the two serve the same artifacts. They do, but pinning the bytes the image actually fetches is the whole point of the hash, and it costs nothing to read them from the one URL that matters. The JDBC and LibreOffice hashes are already taken from their own download URL. * Read the maven-metadata.xml and the WAR from maven.xwiki.org/releases, sharing the one repository URL, and read the cycle POM of 'updateJDBC' from there as well rather than from a second repository. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Check the generated files on Pull Requests too * The Gradlew Check workflow is what guarantees that the committed generated files are what the templates and versions.json produce, but it only ran on a push to master - so a Pull Request changing either was merged before anything verified it, and master was where it broke. * Run it on Pull Requests as well. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Never downgrade the XWiki version of a cycle * Apply to 'updateXWiki' the floor rule 'updateJDBC' already has. A pinned version ahead of everything the Maven metadata lists means the metadata lost it (a release pulled for being broken, a repository half synced), not that the cycle should be rebuilt and republished backwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Smoke test what a change is about, and say why it failed * Boot, for each cycle, the variant of every database whose JDBC driver 'updateJDBC' moved, on top of the postgres-tomcat baseline. A driver bump is the one DB-specific change there is, and nothing else in the build ever connects XWiki to a database - the Docker Build workflow only checks that the jar landed in WEB-INF/lib - so a MySQL connector crossing a major went out entirely unbooted. 'updateJDBC' now reports which database moved, and '-Pvariants' overrides the selection. * Report the tail of the container logs when an instance never comes up, while the containers still exist: the 'down -v' takes them away, and a bare timeout message is not something a failed release can be diagnosed from. * Match the version reported by the REST API between boundaries instead of as a substring. One version string can be the prefix of another, so an instance answering 17.10.12 satisfied a smoke test expecting 17.10.1, i.e. the release gate went green on the wrong image. * Fail when the docker-compose.yml rewrites that isolate an instance find nothing to replace, instead of silently booting on 8080 in the shared project and tearing down, volumes included, whatever was started there by hand. * Tear down the instance's compose project before starting it too, so that a stale one left by a killed daemon or a reboot is not adopted and reported on in place of the image just built. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Keep the official images file honest about what it declares * Stop untracked files blocking 'submitOfficialImage'. What the guard is about is whether the commits it emits are reachable upstream, which a release manager's scratch notes have no bearing on, so it now looks at tracked changes only - and names them, instead of leaving 'git status' to be run by hand. * Define what a cycle 'role' means in a single map, the section comment and the Pull Request label together, and check the model against it there. The labels used to live apart from the check, leaving the Pull Request body free to render an unknown role as 'null' while everything else looked right. * Reject a variant that build.gradle generates but this file does not declare. It would be generated, committed and built by CI, yet never reach the file that publishes it, so the images would simply not exist on DockerHub. * Retry syncing the fork, which GitHub creates asynchronously and which a first-ever run can therefore fail to find. * Drop 'capture's working directory parameter, which no caller passes, and the unused ReplaceTokens import. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Submit the current URL of this repository * Emit 'GitRepo: https://github.com/xwiki/xwiki-docker.git'. The published file still names xwiki-contrib/docker-xwiki, which only resolves through the redirect GitHub kept when this repository moved to the xwiki organisation, and that redirect goes away the day anything else is created under the old name - taking the official image builds with it. This is the one line by which the generated file now differs from the published one. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * submitOfficialImage now refuses to submit images the Docker Build workflow has not built successfully: what docker-library rebuilds is the GitCommit(s) emitted in the library file, so that workflow is what says they build at all, and without the check the first thing to notice a broken Dockerfile is the upstream Pull Request. The workflow only triggers on the paths the images are made of, so a HEAD that triggered no run of its own is checked against the latest run of the release branch instead; no run at all is reported but not fatal. '-PskipCiCheck' waives it, since that workflow also goes red on the transient download failures it is prone to. * Add '-PtargetRepo=<you>/official-images' to submitOfficialImage, so that the write path - the one part a dry run cannot exercise - can be rehearsed against a repository of your own rather than run for the first time during a release. A target you own needs no fork (GitHub does not let you fork your own repository), so the branch then lives in it and the Pull Request is opened inside it; a rehearsal waives the guards on what is reachable upstream, that being the point of it. * Drop 'gradle/*.gradle' from the paths triggering the Docker Build workflow: none of those scripts takes part in the generation, so a change to one cannot change an image, and building the whole matrix for it costs 12 jobs for nothing. That they still configure and generate is what Gradlew Check verifies, now on Pull Requests too. * Move the ports runAll publishes off 8080 (18081 and up, '-PbasePort' to move them), which is what smokeTest already does and for the same reason: 8080 is the port a developer's own instance holds, and binding it fails the boot for a reason that has nothing to do with the image. * Parse the '-Pcycles' / '-Pvariants' selections once, through a 'parseList' helper, instead of repeating the same tokenize/trim/filter in four places. * Only tell the release manager to regenerate the images when something actually moved, rather than after every update task run. * Say in the README (and next to buildImage) that smokeTest and runAll overwrite the local image tag they build, the published official image of that version included, and document the new flags. * XDOCKER-18: Automate updating the stable version * Establish, rather than assume, that a Docker Build run describes the images being submitted. A commit that triggered no run of its own was taken to have changed no image, but a run GitHub has not created yet looks exactly the same over the API - and pushing then submitting a few seconds later is how a release goes - so the check could go green on images nothing had built. The fallback is now the latest run of the history being submitted, and it is accepted only when it contains the GitCommit of every cycle. * Stop the two 'gh api' guards from failing open. Both allowed the call to fail and read the resulting empty output as "nothing found", so a rate-limited or unauthenticated 'gh' passed them silently - and for the open Pull Request one, passing means deleting the branch, which closes the Pull Request the guard exists to protect. * Refuse to emit an empty GitCommit. A cycle directory that has never been committed has no last-touching commit and 'git log' says so by printing nothing, which submitted a 'GitCommit:' line with nothing on it. Adding a cycle leaves exactly such an untracked directory behind. * Run the captured commands in the project directory instead of wherever the build process happens to sit, since all of them ask git about this repository. * Drop the 'run ./gradlew' line that 'updateJDBC' logged whether or not anything had changed, and which repeated the one just above it when something had. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Refuse to generate a library file declaring the same tag twice. The tags a cycle contributes are derived from its version, so two cycles on the same version line both claim it: a stable cycle '18' pinned to 18.4.4 while cycle '18.4' is on 18.4.4 too - the state right after an intermediate LTS is declared, if the stable line has not moved past it yet - makes both emit '18.4', '18.4.4' and every per-variant form of them, which docker-library rejects. Caught at configuration time, where the tags are decided, rather than upstream in front of the docker-library maintainers. * Read a 'merge-base --is-ancestor' that cannot answer as a no rather than as an error, in the Docker Build check. The runs are walked newest first, so a single commit missing from the clone (force-pushed away, never fetched) used to abort a release on a run that was only going to be skipped anyway. It cannot make the check pass on something unbuilt either, since it only ever removes a run from the candidates. * Name the web container after the whole variant in the compose rewrites, which is how the template names it ('xwiki-${db}-tomcat-web' in a '<db>-<servlet>' directory is 'xwiki-<variant>-web'), instead of hardcoding '-tomcat-'. 'smokeTestVariants' deliberately assumes no servlet container, and this now agrees with it. * Validate '-PbasePort' and '-PtargetRepo' instead of coercing them. Both are read at configuration time, so a missing or malformed value used to fail every task of the build with a message naming neither the property nor what it expected - and an invalid '-PtargetRepo' was taken for a rehearsal (it is not the upstream repository) and only failed deep in the write path. * Encode the submitted library file from its UTF-8 bytes rather than from the platform default charset. * Disconnect the smoke test's polling connection in a 'finally': the attempt that throws is the common case there, a boot answering nothing for minutes. * Say in the README that 'stopAll' removes the volumes of the instances it stops, and therefore the wiki content created in them. * XDOCKER-18: Automate updating the stable version * Default 'stream' to running in the project directory, as 'capture' and 'exitCode' already do. A process inherits the working directory of the JVM that starts it, which for a Gradle build is the daemon's and therefore not something a script gets to assume. Every command run through it passes one today, so this only ever matters for the next one written without. * Validate the '-Pcycles' and '-Pvariants' entries against the cycles and variants that exist, the way '-PbasePort' and '-PtargetRepo' already validate theirs. Both ways of getting them wrong were silent: an empty value ('-Pcycles' with nothing after it) left an empty list, which every caller reads as "no selection" and therefore as "all of them" - the opposite of narrowing anything down - and a typo such as '-Pvariants=mysql' surfaced only later as a missing directory, naming a path rather than the property that named it. The message lists the available values, which is what whoever got it wrong is looking for. * Establish the submission guards before '-PdryRun' returns rather than after it. "Am I ready to submit?" is what a dry run is reached for, and answering only "here is what the file would look like" left the release manager to discover a dirty tree, an unpushed HEAD or a red Docker Build run on the real invocation instead. A dry run reports them and still succeeds where a real one fails on them, since the natural moment to preview the file is right after './gradlew release', with the regenerated tree not yet committed. * Collect those reasons instead of throwing on the first, so that one run names everything that is in the way. 'checkImagesBuild' returns its reason rather than raising it, for the same purpose. * Check for a Pull Request still open from the working branch in a rehearsal too. Unlike the guards on what is reachable upstream, that one is about not destroying something on the target, which a target of your own has just as much of. * Say in the Docker Build check why looking at the most recent runs alone is enough for the two questions it asks. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Regenerate the images in 'submitOfficialImage', so that a 'versions.json' pushed without its regenerated directories is reported as a dirty tree instead of publishing tags the pushed Dockerfiles do not produce. * Refuse in 'updateXWiki' a bump that would put two cycles on the same version line, which the duplicate tag check rejects at configuration time and would therefore leave every later './gradlew' failing. * Boot in 'smokeTest' the cycles 'generate' rewrote as well as the ones whose versions moved, so that a 'template/' edit is smoke-tested by 'release' too. * Add '-Pfresh' to 'runAll', to remove each instance and its volumes before starting it. * Validate the boolean command line flags, '-PdryRun=false' having read as true. * Drop a comment justifying itself by an earlier revision, and a transient issue key from another. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Mount the database init script from the path it is generated at ('./mysql/init.sql' and './mariadb/init.sql' rather than './init.sql'). Docker created a directory for the missing source, which the database images then failed to read as a SQL script, so the db container of every mysql and mariadb variant exited at boot and XWiki never came up. Broken since the compose files were reworked; only the postgres variants, which mount no script, were unaffected. * Fail generation when a generated compose file bind mounts a relative path that is not a file next to it, so that the above cannot come back unnoticed: the image build never runs the compose file and 'smokeTest' boots one variant per cycle by default, so nothing else covers the other two. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> * XDOCKER-18: Automate updating the stable version * Replace the raw NUL byte separating the path from the content in 'directorySnapshot' with a '\n' escape. The control byte in the source made git treat build.gradle as binary, so the script holding the whole generation logic was shown as "Binary file not shown" on a Pull Request and could only be merged by taking one whole side instead of as text. * XDOCKER-18: Automate updating the stable version * Drop 'runAll'/'stopAll' and their '-PbasePort'/'-Pfresh' flags: they boot-checked nothing that 'docker compose up' in a variant directory does not already do. * Drop the Docker Build workflow check of 'submitOfficialImage' and its '-PskipCiCheck' waiver: docker-library builds every image on its own Pull Request before merging it, unwaivably, so this only duplicated that gate - and its own waiver existed because the workflow fails on transient downloads often enough. * Drop the '-PtargetRepo' rehearsal mode: it was scaffolding for developing the write path, and it left two modes to read through the whole submission flow. * Drop the configuration-time duplicate tag check: 'updateXWiki' already refuses to put two cycles on the same version line, and it did so at configuration time, where a hand-edited 'versions.json' broke plain './gradlew generate' too. * Drop the generated-directory content snapshots: they read every generated file twice on every './gradlew' to pick the cycles a 'template/' edit changed, which 'smokeTest -Pcycles=...' asks for directly. * Have '-PdryRun' stop before the submission guards rather than collect their verdicts: the guards are about not writing something wrong, and a dry run writes nothing, so a real run now throws on the first problem instead. * Fail 'updateLibreOffice' when the sha256 published for the pinned version no longer matches the pinned one, instead of adopting the new hash and reporting it as an update: different bytes under a version already published is what pinning a hash exists to detect. * Simplify 'compareVersions' and drop the now unused 'exitCode' helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply(a)anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-docker/settings/notifications

1 0

[xwiki/xwiki-docker] efe127: XDOCKER-18: Automate updating the stable version
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/XDOCKER-18-automate-docker-images-release Home: https://github.com/xwiki/xwiki-docker Commit: efe127b9fa42a6c4f93ff5b77d460fabd25f92b6 https://github.com/xwiki/xwiki-docker/commit/efe127b9fa42a6c4f93ff5b77d460f… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M README.md M build.gradle M gradle/helpers.gradle M gradle/official-image.gradle M gradle/run-instances.gradle M gradle/update-versions.gradle Log Message: ----------- XDOCKER-18: Automate updating the stable version * Drop 'runAll'/'stopAll' and their '-PbasePort'/'-Pfresh' flags: they boot-checked nothing that 'docker compose up' in a variant directory does not already do. * Drop the Docker Build workflow check of 'submitOfficialImage' and its '-PskipCiCheck' waiver: docker-library builds every image on its own Pull Request before merging it, unwaivably, so this only duplicated that gate - and its own waiver existed because the workflow fails on transient downloads often enough. * Drop the '-PtargetRepo' rehearsal mode: it was scaffolding for developing the write path, and it left two modes to read through the whole submission flow. * Drop the configuration-time duplicate tag check: 'updateXWiki' already refuses to put two cycles on the same version line, and it did so at configuration time, where a hand-edited 'versions.json' broke plain './gradlew generate' too. * Drop the generated-directory content snapshots: they read every generated file twice on every './gradlew' to pick the cycles a 'template/' edit changed, which 'smokeTest -Pcycles=...' asks for directly. * Have '-PdryRun' stop before the submission guards rather than collect their verdicts: the guards are about not writing something wrong, and a dry run writes nothing, so a real run now throws on the first problem instead. * Fail 'updateLibreOffice' when the sha256 published for the pinned version no longer matches the pinned one, instead of adopting the new hash and reporting it as an update: different bytes under a version already published is what pinning a hash exists to detect. * Simplify 'compareVersions' and drop the now unused 'exitCode' helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-docker/settings/notifications

1 0

[xwiki/xwiki-platform] e247bc: XWIKI-24710: Add script APIs to count distinct act...
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/XWIKI-24710 Home: https://github.com/xwiki/xwiki-platform Commit: e247bc49547e568f6a0980d7b9f80b587f04fd39 https://github.com/xwiki/xwiki-platform/commit/e247bc49547e568f6a0980d7b9f8… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/pom.xml M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/DataManager.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/TooManyExtensionsException.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/DefaultDataManager.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/DatePingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/DistributionPingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/ExtensionPingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/script/ActiveInstallsScriptService.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/it/org/xwiki/activeinstalls2/internal/PingSenderIT.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/DataManagerTest.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/script/ActiveInstallsScriptServiceTest.java Log Message: ----------- XWIKI-24710: Add script APIs to count distinct active installs globally and per extension * Add DataManager#countDistinctInstalls(), counting the distinct instances matching a query with a cardinality aggregation on the instance id. Contrary to countInstalls(), which counts pings, this counts installs: an instance pings once a day but also on every restart. * Add DataManager#countDistinctInstallsByExtension(), returning the same count per extension id in a single query. It uses a reverse nested aggregation since the extensions are indexed as a nested type while the instance id is on the root document. It returns a SequencedMap, so that the documented bucket ordering is part of the type rather than only of the Javadoc. * Expose both through the activeinstalls2 script service. * Both methods are default methods throwing their declared checked exception, so that existing DataManager implementations keep working. Mark them @Unstable: the shape of the query slot and whether the per-extension count should be rights-gated are still open, and an unstable API can be reshaped within the cycle rather than deprecated. * Set the cardinality precision threshold explicitly on both aggregations, to the maximum Elasticsearch supports (40000), so that the counts are exact for the whole instance population. HyperLogLog++ keeps its sparse representation until the observed count approaches the threshold, so the memory a count uses follows that count rather than that bound. Document in both APIs the count above which the result is an approximation. * Bound how many extensions a single per-extension count reports, so that both the point at which the query starts failing and the cost it can impose on the Elasticsearch cluster stay known. * Detect that a per-extension count would report only some of the extensions by asking the terms aggregation for one extension more than the maximum, and refusing the count when that extra one comes back, rather than by reading sum_other_doc_count. The former is exact whatever the number of shards of the index, while the latter also holds the buckets that a shard didn't report because they fell below its shard_size. * Raise the new TooManyExtensionsException rather than a generic Exception when that bound is exceeded, so that a caller can tell that case apart from a failure to reach the data without matching on a message. * Assemble the queried field paths from the mapping property names declared by the ping data providers, instead of re-declaring them and keeping them in sync by hand. DatePingDataProvider now uses the shared instance id path too. * Document that a query on the extensions doesn't restrict the extensions being counted, that the counts are keyed by the id an extension is installed under rather than resolved through its features, and that restricting a count to the instances that are actually active is a matter of querying on the ping date. * Document on countInstalls() that it counts pings and not instances despite its name, and correct the field that its Javadoc example queries, which doesn't exist in the mapping. * Encode the JSON query in UTF-8 rather than in the platform default charset before base64-encoding it into the Elasticsearch wrapper query, and share the wrapper construction between countInstalls() and the search requests. * Extend the integration test to verify the distinct counts with and without a query, and the per-extension counts and their order. Add unit tests for the DataManager default methods and for the script service, and raise the module's instruction coverage ratios to 0.91 with the integration tests and 0.10 without. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-platform/settings/notifications

1 0

[xwiki/xwiki-docker] 260cdb: XDOCKER-18: Automate updating the stable version
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/XDOCKER-18-automate-docker-images-release Home: https://github.com/xwiki/xwiki-docker Commit: 260cdb58f9275d333b81e5ec07e1aaa631e79abd https://github.com/xwiki/xwiki-docker/commit/260cdb58f9275d333b81e5ec07e1aa… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M build.gradle Log Message: ----------- XDOCKER-18: Automate updating the stable version * Replace the raw NUL byte separating the path from the content in 'directorySnapshot' with a '\n' escape. The control byte in the source made git treat build.gradle as binary, so the script holding the whole generation logic was shown as "Binary file not shown" on a Pull Request and could only be merged by taking one whole side instead of as text. To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-docker/settings/notifications

1 0

[jira@xwiki.org][Issue] Updates for XWIKI-24742: The locale of a client-side translation query is ignored
by XWiki Notifications 26 Aug '26

26 Aug '26

1 0

[jira@xwiki.org][Issue] Updates for XWIKI-24741: Provide a localization adapter for ES modules
by XWiki Notifications 26 Aug '26

26 Aug '26

1 0

[xwiki/xwiki-docker] 015e8e: XDOCKER-18: Automate updating the stable version
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/XDOCKER-18-automate-docker-images-release Home: https://github.com/xwiki/xwiki-docker Commit: 015e8efdf5930be8572d0a9b24dc4283edaab9cb https://github.com/xwiki/xwiki-docker/commit/015e8efdf5930be8572d0a9b24dc42… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M 16/mariadb-tomcat/docker-compose.yml M 16/mysql-tomcat/docker-compose.yml M 17/mariadb-tomcat/docker-compose.yml M 17/mysql-tomcat/docker-compose.yml M 18.4/mariadb-tomcat/docker-compose.yml M 18.4/mysql-tomcat/docker-compose.yml M 18/mariadb-tomcat/docker-compose.yml M 18/mysql-tomcat/docker-compose.yml M build.gradle M template/docker-compose.yml Log Message: ----------- XDOCKER-18: Automate updating the stable version * Mount the database init script from the path it is generated at ('./mysql/init.sql' and './mariadb/init.sql' rather than './init.sql'). Docker created a directory for the missing source, which the database images then failed to read as a SQL script, so the db container of every mysql and mariadb variant exited at boot and XWiki never came up. Broken since the compose files were reworked; only the postgres variants, which mount no script, were unaffected. * Fail generation when a generated compose file bind mounts a relative path that is not a file next to it, so that the above cannot come back unnoticed: the image build never runs the compose file and 'smokeTest' boots one variant per cycle by default, so nothing else covers the other two. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-docker/settings/notifications

1 0

[xwiki/xwiki-platform] f485f6: XWIKI-24710: Add script APIs to count distinct act...
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/XWIKI-24710 Home: https://github.com/xwiki/xwiki-platform Commit: f485f60d5c76a026faa7cafdb669c71bdb1ad3d5 https://github.com/xwiki/xwiki-platform/commit/f485f60d5c76a026faa7cafdb669… Author: Vincent Massol <vincent(a)massol.net> Date: 2026-08-26 (Wed, 26 Aug 2026) Changed paths: M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/pom.xml M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/ActiveInstallsConfiguration.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/DataManager.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/TooManyExtensionsException.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/DefaultActiveInstallsConfiguration.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/DefaultDataManager.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/DatePingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/DistributionPingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/internal/data/ExtensionPingDataProvider.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/main/java/org/xwiki/activeinstalls2/script/ActiveInstallsScriptService.java M xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/it/org/xwiki/activeinstalls2/internal/PingSenderIT.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/ActiveInstallsConfigurationTest.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/DataManagerTest.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/internal/DefaultActiveInstallsConfigurationTest.java A xwiki-platform-core/xwiki-platform-activeinstalls2/xwiki-platform-activeinstalls2-api/src/test/java/org/xwiki/activeinstalls2/script/ActiveInstallsScriptServiceTest.java M xwiki-platform-tools/xwiki-platform-tool-configuration-resources/src/main/resources/xwiki.properties.vm Log Message: ----------- XWIKI-24710: Add script APIs to count distinct active installs globally and per extension * Add DataManager#countDistinctInstalls(), counting the distinct instances matching a query with a cardinality aggregation on the instance id. Contrary to countInstalls(), which counts pings, this counts installs: an instance pings once a day but also on every restart. * Add DataManager#countDistinctInstallsByExtension(), returning the same count per extension id in a single query. It uses a reverse nested aggregation since the extensions are indexed as a nested type while the instance id is on the root document. It returns a SequencedMap, so that the documented bucket ordering is part of the type rather than only of the Javadoc. * Expose both through the activeinstalls2 script service. * Both methods are default methods throwing their declared checked exception, so that existing DataManager implementations keep working. Mark them, and the two new configuration methods, @Unstable: the shape of the query slot and whether the per-extension count should be rights-gated are still open, and an unstable API can be reshaped within the cycle rather than deprecated. * Set the cardinality precision threshold explicitly on both aggregations: the maximum Elasticsearch supports (40000) for the global count, and the new activeinstalls2.extensionInstanceCountPrecisionThreshold property (10000 by default) for the per-extension one, which pays that memory once per extension counted. Document in both APIs the count above which the result is an approximation. * Add ActiveInstallsConfiguration#getMaxExtensionCountPerQuery(), configurable with the new activeinstalls2.maxExtensionCountPerQuery property, to bound how many extensions a single per-extension count can report. This makes both the point at which the query starts failing and the cost it can impose on the Elasticsearch cluster configurable. DefaultActiveInstallsConfiguration clamps both properties into the ranges the role documents. * Detect that a per-extension count would report only some of the extensions by asking the terms aggregation for one extension more than the maximum, and refusing the count when that extra one comes back, rather than by reading sum_other_doc_count. The former is exact whatever the number of shards of the index, while the latter also holds the buckets that a shard didn't report because they fell below its shard_size. * Raise the new TooManyExtensionsException rather than a generic Exception when that bound is exceeded, so that a caller can tell that raising the configured maximum is what's needed without matching on a message. It carries the maximum that was exceeded; the property to raise shapes its message but is not exposed, belonging to the ActiveInstallsConfiguration implementation in use rather than to the exception. * Assemble the queried field paths from the mapping property names declared by the ping data providers, instead of re-declaring them and keeping them in sync by hand. DatePingDataProvider now uses the shared instance id path too. * Document that a query on the extensions doesn't restrict the extensions being counted, that the counts are keyed by the id an extension is installed under rather than resolved through its features, and that restricting a count to the instances that are actually active is a matter of querying on the ping date. * Document on countInstalls() that it counts pings and not instances despite its name, and correct the field that its Javadoc example queries, which doesn't exist in the mapping. * Encode the JSON query in UTF-8 rather than in the platform default charset before base64-encoding it into the Elasticsearch wrapper query, and share the wrapper construction between countInstalls() and the search requests. * Extend the integration test to verify the distinct counts with and without a query, the per-extension counts and their order, that a maximum exactly reaching the number of extensions in the index still counts them all, and that exceeding it fails rather than under-reporting. Add unit tests for the DataManager and ActiveInstallsConfiguration default methods, for the script service and for the configuration, and raise the module's instruction coverage ratios to 0.92 with the integration tests and 0.12 without. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-platform/settings/notifications

1 0

[jira@xwiki.org][Issue] Updates for XWIKI-24741: Provide a localization adapter for ES modules
by XWiki Notifications 26 Aug '26

26 Aug '26

1 0

[xwiki/xwiki-platform]
by XWiki Notifications 26 Aug '26

26 Aug '26

Branch: refs/heads/misc-sonar-regex-backtracking Home: https://github.com/xwiki/xwiki-platform To unsubscribe from these emails, change your notification settings at https://github.com/xwiki/xwiki-platform/settings/notifications

1 0