fineweb-edu HuggingFaceFW
๐ FineWeb-Edu 1.3 trillion tokens of the finest educational data the ๐ web has to offer Paper: https://arxiv.org/abs/2406.17557 What is it? ๐ FineWeb-Edu dataset consists of 1.3T tokens and 5.4T tokens (FineWeb-Edu-score-2) of educational web pages filtered from ๐ท FineWeb dataset. This is the 1.3 trillion version. To enhance FineWeb's quality, we developed an educational quality classifier using annotations generated by LLama3-70B-Instruct. Weโฆ See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu.
- Type
- dataset
- License
- odc-by
- Language
- en
- Downloads
- 403,783
- Likes
- 1,258
- Access
- public
- Files
- 0
README
--- license: odc-by task_categories: - text-generation language: - en pretty_name: FineWeb-Edu size_categories: - n>1T configs: - config_name: default data_files: - split: train path: data/*/* features: - name: text dtype: string - name: id dtype: string - name: dump dtype: string - name: url dtypeโฆ