AI詩詞學習步道：多模態學習系統設計
講者/Speaker： 楊承穎, YANG CHENG-YING
本研究以「AI詩詞學習步道」為核心，從國小古詩教學常見的「會背卻不懂、能念卻無感」痛點出發，建構一套適用國小全年級的多模型協作與多模態學習系統。系統整合 Gemini 生成式 AI、規則式問答引擎、圖像辨識、表情辨識、語音互動與聲音分析，讓學生能透過文字提問、語音對話、看圖猜詩、心情配詩、朗讀練習與詩意轉譯，從背誦走向理解、從觀看走向表達、從接受答案走向主動創作。研究建置 83 首詩詞資料庫、20 類意象、5 類情緒標籤與 15 類問答意圖，並以 Top-1 評估完成表情推薦命中率 86.9%、物品意象推薦命中率 80.5%。課堂實作回收問卷，顯示學生肯定系統易用性、朗讀支架與學習動機。本案不只是 AI 工具展示，而是將校本詩詞教材轉化為可互動、可驗證、可推廣的智慧語文學習模式，讓 AI 成為陪伴學生走進詩、理解詩、說出詩與創造詩的學習夥伴。
This project, AI Poetry Learning Pathway, addresses a common challenge in elementary poetry instruction: students can often recite classical poems, yet struggle to visualize poetic imagery, understand emotions, and express meaning in their own words. To transform poetry learning from memorization into interaction, this project develops a multimodal AI learning system for elementary students across grade levels. The system integrates Gemini, a rule-based question-answering engine, image recognition, facial expression recognition, speech interaction, and audio analysis, enabling students to learn through asking, listening, speaking, viewing, reading, interpreting, and creating.
The system includes a structured database of 83 classical poems, 20 imagery categories, 5 emotion labels, and 15 question-intent types. It supports multiple learning tasks such as voice-based Q&A, image-to-poem matching, mood-based poem recommendation, reading practice, plain-language interpretation, and creative poetry rewriting. Evaluation results show that the system achieved an 86.9% Top-1 accuracy in emotion-to-poem recommendation and an 80.5% Top-1 accuracy in object-to-imagery poem recommendation. In classroom implementation, student questionnaires indicated positive responses toward system usability, reading support, and learning motivation. This project is not merely a demonstration of AI tools; it transforms school-based poetry materials into an interactive, verifiable, and scalable learning model. By positioning AI as a learning partner rather than an answer generator, the system helps students enter, understand, express, and recreate poetry through meaningful human-AI collaboration.
- 論文全文：AI詩詞學習步道：多模態學習系統設計
- 影像： 相關影片