Labeling Kenyan Social Media Data for a Sentiment Analysis Tool

Volunteering

Overview

Annotate dataset of Kenyan social media text for sentiment analysis.

You have:

• Fluency in English and Kiswahili sufficient to accurately interpret meaning, tone and emotion in written social media text.
• Demonstrated command of Kenyan Sheng and current social media language, including slang, code-switching and online expression.
• The ability to stay focused and objective across repetitive tasks and to apply category definitions consistently over a large number of short texts.
• Comfort learning and using a web-based annotation platform (Label Studio) and managing datasets in Microsoft Excel.
• The ability to complete a defined deliverable within the assignment timeframe.
• Prior experience in data annotation, linguistics or NLP projects is an added advantage but is not required.

Contract

This is a UNV contract. More about UNV contracts.

The Regional Centre of Competence (RCoC) for Digital and AI Skilling — a partnership between the Government of Kenya, Microsoft and the Kenya School of Government, supported by UNDP Kenya — is developing a Sentiment Analysis tool adapted to Kenya's multilingual digital landscape. Standard sentiment models are trained largely on structured, global-English datasets and miss the meaning carried by local expression, code-switching and slang. To close this gap, the RCoC is fine-tuning the XLM-RoBERTa base model on Kenyan social media data so that the tool can read sentiment and emotion accurately across English, Kiswahili and Sheng. This work requires a high-quality, human-labeled dataset to serve as the ground-truth training data for the tool. Online Volunteers will produce this dataset. Through the assignment, volunteers gain practical, portfolio-ready experience in natural language processing (NLP) data annotation, inter-annotator quality methods, and applied AI for the public good, while contributing directly to a locally relevant and responsibly built AI tool for the Government of Kenya. The assignment advances SDG 9 by strengthening innovation and digital capabilities suited to local needs. Volunteers will receive a certificate of appreciation upon successful completion.

We are looking for four (4) Online Volunteers to annotate a dataset of Kenyan social media text so it can be used as ground-truth training data for the RCoC's Sentiment Analysis tool. The dataset include 10,000 data rows and will be cross-checked to verify accuracy and ensure high confidence in the data. The assignment is carried out under the overall supervision of the RCoC Project Lead, while day-to-day guidance on the annotation task is provided by the Junior AI Solution Developers under the RCoC. Each Online Volunteer will: • Attend an initial onboarding session and complete a test run of 50–100 rows to align on the guidelines and category definitions before the annotation period begins. • Independently annotate the complete dataset of 10,000 rows, performing: – Sentiment Annotation: categorizing the overall sentiment of each text snippet into the designated categories (e.g., Positive, Negative, Neutral). – Emotion Classification: labeling the specific emotion conveyed in the text based on the predefined set of categories (e.g., Joy, Anger, Sadness, Fear, Disgust, Surprise). – Contextual Interpretation: accurately interpreting and tagging slang, code-switching and context-specific phrasing heavily used in Kenyan digital spaces (specifically English, Kiswahili and Sheng blends). • Participate in quality assurance through overlap-labeling: Inter-Annotator Agreement (IAA) is measured across every row to ensure high confidence in the data quality. • Conduct all labeling directly within Label Studio (for web-based collaborative tagging) or Microsoft Excel (for offline batch processing), as directed by the project team, following the provided annotation guidelines and category definitions. Deliverables (per Online Volunteer) 1. Completed calibration test run of 50–100 rows and confirmed alignment on the annotation guidelines and category definitions. 2. One complete and accurately annotated set of all 10,000 rows, labeled for sentiment and emotion in adherence with the provided annotation guidelines and category definitions. 3. A complete data export from Label Studio, or an updated Microsoft Excel sheet, fully formatted according to the project's data schema. Workload: at 16–20 hours per week over the 4-week period, each volunteer independently annotates the full 10,000-row dataset.

Selection is based solely on the skills and competencies needed to perform the task. Candidates should have: • Language competency: fluency in English and Kiswahili sufficient to accurately interpret meaning, tone and emotion in written social media text. • Kenyan digital-language familiarity: demonstrated command of Kenyan Sheng and current social media language, including slang, code-switching and online expression, sufficient to interpret informal Kenyan digital communication accurately. • Attention to detail: the ability to stay focused and objective across repetitive tasks and to apply category definitions consistently over a large number of short texts. • Digital-tool proficiency: comfort learning and using a web-based annotation platform (Label Studio) and managing datasets in Microsoft Excel. • Reliability: the ability to complete a defined deliverable within the assignment timeframe. Prior experience in data annotation, linguistics or NLP projects is an added advantage but is not required.

Apply now

Potential interview questions

Added 23 days ago - Updated 14 days ago - Source: unv.org