AI Folks
  • Courses
  • Web Of AI
  • Enterprise
Log in

AI & ML Engineering · NLP and Transformers · Lesson 2 of 6

Tokenisation

How text becomes tokens, and why subword tokenisers won.

⠋░░░░░░░░░░░░░░░░░░░░░░░░Loading the lesson

Previous lessonText PreprocessingNext lessonWord Embeddings