eTA (Electronic Teaching Asistant)

Overview

eTA (Electronic Teaching Assistant) is an interactive application designed to enhance the learning experience by providing personalized assistance to students. Leveraging advanced technologies like OpenAI’s GPT models and LangChain, eTA offers a seamless platform where users can upload course materials, ask questions, and receive detailed answers enriched with multimedia content.

Features

  • Interactive Q&A Interface: Engage in a conversational manner to ask questions about coursework or uploaded documents.
  • Document and Image Processing: Upload PDFs and images; the app extracts text using OCR technology for analysis.
  • Multimedia Content Integration:
    • Related Images: Fetches relevant images using Google’s Custom Search API to provide visual context.
    • Related Videos: Searches YouTube for videos related to your query, extracts transcripts, and directs you to the exact timestamp where your topic is discussed.
  • External Resource Integration:
    • Piazza Integration: Connects with Piazza to fetch and process class discussions.
    • Stack Overflow Search: Retrieves relevant questions and answers from Stack Overflow to supplement technical queries.
  • Customizable Settings:
    • Choose between different GPT models (e.g., GPT-4, GPT-3.5-Turbo).
    • Toggle options to display related images or videos.
    • Enable or disable Stack Overflow search and Piazza integration.

How It Works

  1. User Interaction: Start by uploading documents or images related to your coursework. Then, ask any question through the chat interface.
  2. Content Processing: The app processes your uploads, extracting text and relevant information for context.
  3. Query Handling:
    • Utilizes OpenAI’s GPT models to generate accurate and context-rich answers.
    • If enabled, fetches related images and videos to enhance the response.
    • Searches Stack Overflow for additional insights on technical questions.
  4. Response Generation: Presents a comprehensive answer, enriched with multimedia content and external resources, all within the chat interface.

Technologies Used

  • Programming Language: Python
  • Frameworks & Libraries:
    • Streamlit: For building the interactive web application.
    • LangChain: Managing language models and embeddings.
    • PyPDF2 & pytesseract: Extracting text from PDFs and images.
    • OpenAI API: Leveraging GPT models for natural language understanding and generation.
    • Google APIs: Integrating Google Custom Search and YouTube Data API.
    • Piazza API: Fetching class discussions.
    • StackAPI: Interacting with Stack Overflow data.
  • Others: requests, BeautifulSoup, YouTubeTranscriptApi, FAISS for vector storage.

Code Highlights

Text Extraction from PDFs

pythonCopy codedef get_pdf_text(pdf_docs):
    text = ""
    for pdf in pdf_docs:
        pdf_reader = PdfReader(pdf)
        for page in pdf_reader.pages:
            text += page.extract_text() or " "
    return text

Analyzing YouTube Transcripts

pythonCopy codedef analyze_transcript_for_timestamp(video_id, query):
    transcript = YouTubeTranscriptApi.get_transcript(video_id)
    transcript_text = " ".join(
        [f"[{entry['start']}] {entry['text']}" for entry in transcript]
    )
    # Use OpenAI API to find when the topic is discussed
    # ...
    return timestamp

Displaying Multimedia Content

pythonCopy codedef display_multimedia_content(answer_text, show_images, show_videos):
    image_html, video_html = "", ""
    if show_images:
        image_urls = search_images(answer_text)
        # Generate HTML to display images
    if show_videos:
        video_info = search_youtube_videos(answer_text)
        # Generate HTML to display video thumbnails and links
    return image_html, video_html

User Interface

  • Sidebar Settings: Customize your experience by selecting the GPT model, enabling image/video search, and integrating Piazza or Stack Overflow content.
  • Chat Interface: A user-friendly chat window where you can type your questions and receive detailed answers.
  • Multimedia Display: Inline display of images and videos related to your query for a richer understanding.

Piazza Integration

For students enrolled in courses using Piazza, eTA can fetch the latest class discussions and answers, integrating them into the assistant’s knowledge base for more course-specific assistance.

Stack Overflow Search

When tackling technical questions, eTA can pull relevant Q&A pairs from Stack Overflow, providing you with tried-and-tested solutions from the developer community.

eTA bridges the gap between students and the vast resources available online, bringing them together in a single, interactive platform. By combining advanced AI with multimedia content and external integrations, it offers a unique and personalized learning assistant.