{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Syntactic Parsing with spaCy\n",
    "## NLP Course - Lab 4: Constituency & Dependency Analysis\n",
    "\n",
    "### Introduction\n",
    "This tutorial is designed to explore the structural analysis of natural language. We will cover:\n",
    "1. **Dependency Parsing**: Focusing on the relationships between individual words.\n",
    "2. **Constituency Parsing**: Focusing on the hierarchical phrase structure.\n",
    "\n",
    "### Prerequisites\n",
    "You will need `spacy` and `benepar` (Berkeley Neural Parser)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Setup and Environment\n",
    "We will use a transformer-optimized model for spaCy and integrate `benepar` for constituency parsing, as spaCy does not provide this out-of-the-box."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install spacy benepar\n",
    "import spacy\n",
    "from spacy import displacy\n",
    "import benepar\n",
    "\n",
    "# Download models\n",
    "spacy.cli.download('en_core_web_md')\n",
    "benepar.download('benepar_en3')\n",
    "\n",
    "# Load the spaCy model and add the benepar component to the pipeline\n",
    "nlp = spacy.load('en_core_web_md')\n",
    "if 'benepar' not in nlp.pipe_names:\n",
    "    nlp.add_pipe('benepar', config={'model': 'benepar_en3'})\n",
    "\n",
    "print('\\nPipeline components:', nlp.pipe_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Dependency Parsing\n",
    "Dependency parsing identifies the **head** words and their **dependents**. In Computer Science terms, this results in a **Directed Acyclic Graph (DAG)**.\n",
    "\n",
    "#### 2.1 Visualization\n",
    "We use `displacy` to view the grammatical dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = nlp('The experienced computer science students are analyzing complex linguistic structures.')\n",
    "\n",
    "# Visualize the dependency tree\n",
    "displacy.render(doc, style='dep', jupyter=True, options={'distance': 120})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 Navigating the Dependency Graph\n",
    "Every `Token` in a spaCy `Doc` has attributes that allow you to traverse the tree: `.head`, `.children`, `.lefts`, and `.rights`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"{'Text':<12} | {'Dep':<10} | {'Head':<12} | {'Children'}\")\n",
    "print('-' * 60)\n",
    "\n",
    "for token in doc:\n",
    "    children = [child.text for child in token.children]\n",
    "    print(f\"{token.text:<12} | {token.dep_:<10} | {token.head.text:<12} | {children}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Constituency (Phrase Structure) Parsing\n",
    "Constituency parsing breaks sentences into sub-phrases (constituents). This follows Context-Free Grammar (CFG) rules.\n",
    "\n",
    "#### 3.1 Inspecting the Tree Structure\n",
    "With `benepar`, we can access labels like `S` (Sentence), `NP` (Noun Phrase), and `VP` (Verb Phrase)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentence = 'The student solved the problem with a clever algorithm.'\n",
    "doc = nlp(sentence)\n",
    "sent = list(doc.sents)[0]\n",
    "\n",
    "def print_tree(node, depth=0):\n",
    "    # Extract labels from the benepar extension\n",
    "    label = node._.labels[0] if node._.labels else 'TOKEN'\n",
    "    print('  ' * depth + f'[{label}] {node.text}')\n",
    "    for child in node._.children:\n",
    "        print_tree(child, depth + 1)\n",
    "\n",
    "print_tree(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2 Handling Ambiguity\n",
    "Syntactic ambiguity is a major challenge in NLP. Observe how the parser handles Prepositional Phrase (PP) attachment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ambig_1 = nlp('He ate the salad with a fork.')\n",
    "ambig_2 = nlp('He ate the salad with croutons.')\n",
    "\n",
    "print('--- Sentence 1 Tree ---')\n",
    "print_tree(list(ambig_1.sents)[0])\n",
    "print('\\n--- Sentence 2 Tree ---')\n",
    "print_tree(list(ambig_2.sents)[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Comparison Summary\n",
    "\n",
    "| Feature | Dependency | Constituency |\n",
    "| :--- | :--- | :--- |\n",
    "| **Focus** | Word-to-word relations | Hierarchical grouping |\n",
    "| **Application** | Information Extraction | Grammar Checking |\n",
    "| **Logic** | Head/Dependent | Part/Whole |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
