📚 CBSE Grade-10 Study Guide
HomeArtificial Intelligence (417)

Artificial Intelligence (417) — CBSE Class 10 board question

Q1. [1] § Test Yourself and Reflection Time
______ is a term used for any word or number or special character occurring in a sentence. (Token / Punctuator)
Previously asked in CBSE board exam
2023 104 Q5 (i)
Generated by claude-sonnet-4-6 · 2026-06-21 03:18 · grounding rag
Model Answer

Token is a term used for any word or number or special character occurring in a sentence.

Source: Chapter 6, Section 6.3 / Text Normalisation – Tokenisation

Explanation

The examiner expects the single correct term "Token." In tokenisation (a step of Text Normalisation in NLP), the text is broken into smaller units called tokens — these can be words, numbers, or special characters. Do not write "Punctuator," as that refers only to punctuation marks, not all types of units in a sentence.

If a question refers to an image, map, graph or diagram that is not shown here, find the actual CBSE question paper on the CBSE website: cbse.gov.in.
Previous-year CBSE Grade 10 board exam questions, organised by subject and chapter, each with a model answer — free to read and print.