[read_QC_trim] Hard failure occurs when 0 reads are detected, soft failure is desired

[xonq]

🐛

📝 Describe the Issue

Terra will red X (hard fail) when 0 reads are detected. Currently, this stops at "fastq_scan_clean", but would likely fail downstream if that issue is resolved. It is desirable that a soft failure would occur akin to if there was > 0 reads, but less than the threshold.

🔁 How to Reproduce

Run TheiaProk_Illumina_PE or any other workflow using read_QC_trim / independently dispersed tasks (e.g. TheiaViral) with a spoofed FASTQ and watch it fail. This should also be done with a FASTQ that has reads initially, but would fail if the reads are cleaned, i.e. it passes through the raw steps but fails the clean steps

💻 Version Information

<= v4.1.0

[MrTheronJ]

I think this was mostly addressed in PR #1036. I'm not seeing a hard failure in TheiaProk_Illumina_PE when using an empty fastq file. However, TheiaViral will still need to be updated to mirror the same behavior.

My thoughts:

• Right now we only have a clean_check_reads task after calling rasusa. I think it may also make sense to add a raw_check_reads at the very beginning of the workflow, even if it's only checking for completely empty fastqs.

• I also think it might make sense to add a check in the fastq_scan task to return 0 reads when the input fastqs are empty rather than erroring out entirely. However, in the case with TheiaViral, downstream tasks like rasusa would still hard fail before reaching the clean_check_reads task.

• Alternatively, task_screen.wdl is currently doing quite a lot already. We could potentially split this out into a few simpler/more dedicated tasks where one could exclusively check whether a fastq is empty and return a simple yes/no result.

Thoughts?

[xonq]

The issue may not be reproduced via a spoofed fastq, though it was highlighted in theiaprok_illumina_pe (sample NC_ext + NTC_PCR) as an extant problem related to some clean FASTQ errors - specifically a null mean_length value. The overall sentiment of the issue stands for read_qc_trim

[MrTheronJ]

In that example, it looks like those samples hard failed because skip_screen was set to True. Is the goal to support soft failures in cases where task_screen is skipped as well?

[xonq]

I think soft failures should occur in all use-cases for these workflows when it has to do with read quality, in accord with the paradigm we are using