This course bridges statistical NLP, deep neural sequence modeling, and modern Large Language Models (LLMs). The course equips students with mathematical theory and hands-on PyTorch skills to build, evaluate, and deploy production NLP systems from first principles. Learning Outcomes: 1. Foundations: Analyze discrete text signals, linguistic ambiguity (6 tiers), and statistical power laws (Zipf, Heaps) to build robust Unicode NFKC pipelines. 2. Algorithmic Mastery: Implement core algorithms from scratch: BPE subword tokenizers, smoothed N-gram LMs, dynamic programming Viterbi POS taggers, and Shift-Reduce dependency parsers. 3. Neural Sequence Models: Build and train deep neural networks in PyTorch, including Bengio (2003) neural LMs, custom 4-gate LSTM cells, and Seq2Seq with Bahdanau attention. 4. Transformers & LLMs: Implement Multi-Head Attention from raw tensor math, fine-tune pre-trained BERT models, build Vector RAG pipelines, and execute LoRA fine-tuning. 5. Systems & Safety: Benchmark models (Perplexity, F1, BLEU), audit demographic fairness parity, and deploy multi-tier AI safety guardrails and INT8 quantization.

1. Primary Textbook (指定教科書 / Required Textbook)

a. Kristiani, Endah. (2026). Applied Natural Language Processing: A Hands-on Engineering Approach with PyTorch and Transformers (Course Notes & Lab Manual). Tunghai University. (Provided in PDF).

b. Jurafsky, Daniel, & Martin, James H. (2024). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (3rd ed. draft). Stanford University / Pearson.
Online Free Access: https://web.stanford.edu/~jurafsky/slp3/

2. Recommended Reference Books (參考書籍 / Reference Materials)

a. Tunstall, Lewis, von Werra, Leandro, & Wolf, Thomas. (2022).
Natural Language Processing with Transformers: Building Language Applications with Hugging Face (Revised 2nd ed.). O'Reilly Media. ISBN: 978-1098136796.

b. Eisenstein, Jacob. (2019).
Introduction to Natural Language Processing. MIT Press. ISBN: 978-0262042840.

c. Goldberg, Yoav. (2017).
Neural Network Methods in Natural Language Processing (Synthesis Lectures on Human Language Technologies). Morgan & Claypool Publishers. ISBN: 978-1627052986.

d. Vaswani, Ashish, et al. (2017).
"Attention Is All You Need." Advances in Neural Information Processing Systems (NeurIPS 2017).

e. Hu, Edward J., et al. (2021).
"LoRA: Low-Rank Adaptation of Large Language Models." arXiv preprint arXiv:2106.09685.

f. Lewis, Patrick, et al. (2020).
"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." NeurIPS 2020.

3. Online Resources & Documentation (線上學習資源與工具文件)

a. PyTorch Official Tutorials & Documentation: https://pytorch.org/tutorials/

b. Hugging Face Transformers & Datasets Documentation: https://huggingface.co/docs

c. LangChain & LlamaIndex Official Documentation: https://docs.langchain.com / d.https://docs.llamaindex.ai

e. Stanford CS224N: Natural Language Processing with Deep Learning: https://web.stanford.edu/class/cs224n/