AI-Powered Proposal Generation for Social Initiatives Incubator Arweqah

Mar 15, 2022

Read more

Teal Flower

1. Problem Statement

Arweqah, a social initiatives incubator, faced significant challenges in managing a high volume of grant applications required to support its numerous startups and social enterprises. The grant application process was both labor-intensive and time-consuming, with each proposal needing to be meticulously crafted to meet specific grant requirements. Given the repetitive nature of content across various proposals, the process often involved multiple teams and could take weeks or even months to complete. Additionally, proposals needed to be in both English and Arabic, presenting a challenge due to the primary focus of most advanced AI tools on English content.

Arweqah required a solution that could:

Automate repetitive content generation while allowing for personalization specific to each grant application.

Generate high-quality bilingual proposals that are aligned with the specific RFP requirements and Saudi Vision 2030.

Integrate visual ideation to streamline the design process, reducing the burden on designers.

2. Scoping and Approach

In response to these challenges, we undertook a comprehensive scoping process to identify the precise needs of Arweqah. This analysis revealed several key areas for intervention:

Volume and Repetition: The need for a system that could handle repetitive content while allowing for customization based on different grant applications.

Personalization: Integration of company-specific information, alignment with Saudi Vision 2030, and adherence to RFP specifics were crucial for creating tailored proposals.

Language Barriers: High-quality Arabic content was essential, necessitating advanced translation capabilities.

Design Integration: Streamlining the design process to include visual elements that complemented the proposal content.

Our approach involved developing an AI-powered tool designed to address these pain points effectively. The solution was designed with the following components:

3. System Architecture

3.1 High-Level Architecture

The system follows a client-server architecture with these main components:

Web-based User Interface: For document uploads and proposal management.

Application Server: Handles request processing and task coordination.

AI Processing Engine: Manages content generation and translation.

Data Storage System: Stores RFP documents, company data, and generated proposals.

External AI Services: Utilizes APIs for advanced text generation and processing.

3.2 Component Diagram

The component diagram illustrates the interaction between the user interface, application server, AI processing engine, data storage, and external services.

4. Detailed System Design

4.1 User Interface (UI) Layer

Technology: Streamlit

Responsibilities:

• Provide an interface for uploading RFP documents.

• Display proposal generation progress.

• Allow users to review and edit generated proposals.

• Facilitate the download of final proposals.

4.2 Application Server

Technology: Python

Key Components:

Request Handler: Manages incoming user requests.

Task Queue: Coordinates proposal generation tasks.

Data Preprocessor: Prepares RFP and company data for AI processing.

Proposal Assembler: Compiles generated content into the final proposal format.

4.3 AI Processing Engine

Technologies: LangChain, Custom AI models

Key Components:

Prompt Engineering Module: Designs dynamic prompts based on RFP requirements.

Content Generation Module: Utilizes RAG to generate proposal content.

Translation Module: Handles English to Arabic translation.

4.4 Data Storage System

Technologies: Pinecone, Local File System

Components:

Vector Database: Stores embedded company and portfolio data.

Document Store: Manages RFP documents and generated proposals.

4.5 External AI Services Integration

Services: OpenAI GPT-4, Anthropic Claude, Groq Cloud

Integration Points:

Text Generation API Calls

Fast Inference Processing

5. Data Design

5.1 Data Model

RFP Document: Stores extracted information from uploaded RFPs.

Company Portfolio: Contains vectorized data about company expertise and past projects.

Generated Proposal: Represents the structure and content of AI-generated proposals.

5.2 Data Flow Diagram

The data flow diagram outlines how data moves through the system, from document upload and parsing to proposal generation and storage.

6. Interface Design

6.1 User Interface Design

Upload Page: Simple interface for RFP document upload.

Generation Progress Page: Displays real-time progress of proposal generation.

Review and Edit Page: Allows users to review and modify generated content.

Download Page: Facilitates retrieval of final proposal documents.

6.2 API Design

RFP Processing API: Handles document upload and information extraction.

Proposal Generation API: Manages the AI-driven content creation process.

Document Retrieval API: Allows access to stored proposals and related data.

7. Processing Design

7.1 Proposal Generation Pipeline

RFP Upload and Parsing

Information Extraction and Vectorization

Prompt Engineering

Content Generation using RAG

Translation (if required)

Proposal Assembly and Formatting

7.2 AI Model Integration

Model Selection Logic: Determines the appropriate AI model based on task requirements.

Error Handling and Retry Mechanism: Ensures robustness in AI service interactions.

8. Performance Considerations

8.1 Scalability

Horizontal Scaling: Application servers scale horizontally to handle increased load.

Caching Mechanisms: For frequently accessed data.

8.2 Optimization

Parallel Processing: Proposal sections are processed in parallel.

Efficient Vectorization: Techniques for efficient vectorization and retrieval of company data.

9. Future Considerations

9.1 System Expansion

Integration with Proposal Management Systems: To further streamline the proposal lifecycle.

Development of a SaaS Offering: For other companies seeking similar solutions.

9.2 AI Enhancements

Continuous Improvement: Feedback loops for AI model enhancement.

Expansion of Language Support: Beyond English and Arabic to include additional languages.

10. Appendices

10.1 Technology Stack Summary

Frontend: Streamlit

Backend: Python

AI/ML: LangChain, Custom Models, OpenAI GPT-4, Anthropic Claude

Data Storage: Local File System

External Services: LlamaParse

Why Our Approach

1. Long Content Generation:

Our multi-step approach enables the generation of comprehensive content exceeding 100 pages, ensuring detailed and thorough proposals.

2. Custom Tool Definition:

The system defines custom tools that retrieve specific information about the company portfolio, the RFP document, and Saudi Vision 2030. These tools act as specialized knowledge bases that the AI can query as needed.

3. Structured Scope of Work:

A predefined scope of work structure covers all essential aspects of a comprehensive proposal, ensuring consistency and completeness in the generated proposals.

4. Dynamic Agent Creation:

The system creates an agent executor that utilizes the defined tools and follows a custom prompt, allowing for flexible and context-aware information retrieval and processing.

5. Multi-Step Proposal Generation:

RFP Summary Generation: Generates a concise summary of the RFP for clear understanding of project objectives.

Vision 2030 Alignment: Extracts relevant information from Saudi Vision 2030, aligning the proposal with national goals.

Section-by-Section Generation: Creates each section of the proposal individually for detailed content creation.

6. Bilingual Content Creation:

Automatic translation of generated content from English to Arabic ensures accessibility to a wider audience and compliance with local requirements.

7. Quality Control Mechanisms:

Multiple retry attempts for each generation step ensure robustness and reliability in content creation.

8. Flexible Content Structuring:

The system generates both the table of contents and detailed content for each section, resulting in well-structured and comprehensive proposals.

9. Integration of Multiple Information Sources:

Combines information from the RFP, company profile, and Saudi Vision 2030 to create tailored, relevant proposals aligned with broader national objectives.

The Challenging Aspects:

1. Multiple-Steps AI Approach:

Leverages LLMs to generate comprehensive content in multiple stages, resulting in formatted content exceeding 100 pages.

2. Tool-Augmented AI:

Custom tools enhance AI’s ability to access specific information on demand, improving content relevance and accuracy.

3. Structured Yet Flexible Generation:

A predefined scope of work ensures comprehensive coverage while dynamic content generation allows for customization for each RFP.

4. Automatic Alignment with National Vision:

Integration of Saudi Vision 2030 ensures proposals align with broader national goals, making them more compelling.

5. Bilingual Automation:

Streamlines the process of creating proposals in multiple languages, saving time and ensuring consistency.

6. Robustness Through Retry Mechanisms:

Enhances reliability and reduces the need for human intervention through robust retry mechanisms.

Outcomes

The implementation of the AI-powered proposal generation tool led to substantial improvements for Arweqah:

Time Savings: Reduced proposal preparation time by over 90%, transforming a process that previously took weeks into one that can be completed in days.

Cost Efficiency: Significant cost savings due to automation of content generation and visual ideation, potentially saving thousands of dollars.

Enhanced Quality and Personalization: Improved proposal quality and personalization, increasing the likelihood of securing grants and effectively supporting social initiatives.

Improved Arabic Content Generation: High-quality Arabic content generation addressed language barriers and ensured proposals were effective in both English and Arabic.

Streamlined Design Process: Automated visual ideation reduced the time and effort required from designers.

Continue Reading

The latest handpicked blog articles