Release/v1.5.4 - #1010
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request bumps the version of data-juicer to v1.5.4 and updates the release notes in both README.md and README_ZH.md. The release highlights include Batch-local Stage Fusion, ARM64 installation support, robustness fixes, and expanded test coverage. There are no review comments, so I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…rt crash" This reverts commit 952dc5e.
…ocessor Remove modelscope from audio dependencies — it monkey-patches transformers' from_pretrained, causing AutoProcessor to download full model weights (~14GB for Qwen-7B-Chat) instead of just tokenizer files, breaking text_chunk_mapper CI tests. SenseVoiceSmall already uses hub="hf" so modelscope is no longer needed. Also add AutoTokenizer.from_pretrained as a fallback step in get_processor() between dashscope and AutoProcessor, which is a more appropriate loader for tokenizer-only use cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
2 participants
Release v1.5.4.
1.5.4README.md/README_ZH.mdwith v1.5.4 release notesRebased on the latest
main(includes #938, #979, #1013). Verified imports anddj-process --helpsucceed with all new OPs.Full release notes will be published on the GitHub Releases page.
Full Changelog: v1.5.3...v1.5.4