HWcloud

See also the lecture

For both tasks, submit your source code and the result, when run on whole dataset (gs://mad-2022-public/). The code is expected to use the Apache beam framework and Dataflow presented in the lecture. Submit directory is /submit/cloud/

Task A

Count the number of occurrences of each 4-mer in the provided data.

Task B

Count the number of pairs of reads which overlap in exactly 30 bases (end of one read overlaps beginning of the second read). You can ignore reverse complement.

Hints:

• Try counting pairs for each 30-mer first.
• You can yield something structured from Map/ParDo operatation (e.g. tuple).
• You can have another Map/ParDo after CombinePerKey.
• Run code locally to quickly iterate and test :)