Exploring the New Features of Crew AI: A Comprehensive Tutorial

In this updated Crew AI tutorial, viewers will learn how to utilize Crew AI's new features to construct an AI newsletter from scratch. The tutorial aims to showcase the process of creating a newsletter that aggregates the latest AI articles from the internet into a visually appealing format similar to popular Twitter accounts. The video promises insights into Crew AI's latest features and practical applications in the real world.

The video delves into the four major updates of Crew AI, prompted by the release of a new version by the creator on X. The updates focus on enhancing reliability and introducing new features. The key updates include support for hierarchical processes, asynchronous task execution, working with expected outputs, and callbacks. These updates aim to streamline processes, improve outcomes, and offer a more robust user experience.

Crew AI now supports hierarchical processes, introducing a chain of command within crews. This structure involves a manager agent overseeing and coordinating tasks among crew agents, ensuring validation of outcomes. The hierarchical approach aims to enhance coordination, consistency, and overall results compared to the previous sequential workflow. Implementing this change involves updating the crew to be hierarchical and setting up a manager LL to orchestrate crew agents effectively.

A significant update in Crew AI allows for asynchronous task execution, enabling parallel processing of multiple tasks. This update revolutionizes task execution efficiency by running tasks concurrently, reducing overall processing time significantly. The example of writing an AI research paper illustrates the time-saving benefits of executing tasks in parallel, leading to faster results and improved productivity in Crew AI workflows.

To use the new asynchronous feature in Crew AI, set async execution to true on the task. You can aggregate outputs of asynchronous tasks by setting the context on another task.

Crew AI now allows adding expected outputs for tasks, enhancing task quality and reliability. For example, in a list ideas task, specifying five bullet points for an article as the expected output avoids confusion.

Callbacks in Crew AI are functions triggered when tasks finish running. For instance, in an idea generation task, a callback can print results, send emails, or save ideas to a CSV file.

The tutorial focuses on four major new features in Crew AI: asynchronous tasks, expected outputs, callbacks, and building a newsletter. A complete crash course for beginners is also available for more information on Crew AI.

The tutorial proceeds to build a new Crew AI newsletter from scratch, emphasizing practical implementation of the discussed features. The process involves creating folders, setting up dependencies, and developing agents, tasks, and tools.

The speaker discusses the importance of defining dependencies for a project, highlighting the need for Python, crew AI version 0.11 or higher, and environment variables. They mention the process of creating a main file to run the project and outline the steps of setting up agents, tasks, and tools in a specific order.

The speaker demonstrates the use of Poetry to install project dependencies by running 'poetry install --no-root' in the terminal. They emphasize the convenience of this method in managing dependencies and setting up projects efficiently.

The speaker confirms the successful setup of the project by opening a new Poetry shell and running a test script. They show the output indicating the project name and verify that the environment is correctly configured for further development.

The speaker begins setting up agents for Crew AAI by creating an 'agents.py' file. They explain the role of the editor agent in overseeing the newsletter creation process and mention setting a maximum iteration limit to prevent endless loops. The speaker plans to include four different agents to run the program efficiently.

The new features in newsletter creation include robustness in output after multiple iterations and the ability for agents to delegate work to other agents. This allows for efficient information retrieval and compilation for newsletters.

The system consists of different agents such as the news fetcher, news analyzer, and news compiler. The news fetcher searches for top AI news stories, the news analyzer provides detailed summaries of articles, and the news compiler consolidates content into a cohesive newsletter.

To resolve import errors, updating the Visual Studio code to use the new Python environment created is necessary. This involves copying the address of the environment, updating the interpreter path in Visual Studio code, and ensuring that all dependencies are installed.

Tasks for agents are created by making a task.py file and defining classes for each task. The tasks involve fetching AI news stories asynchronously, using expected stories to specify output requirements, and implementing features like titles, URLs, and summaries for each story.

The process begins with grabbing news stories and ensuring at least five well-formatted articles are obtained. Asynchronous tasks are used to analyze multiple news articles simultaneously. Context is provided to feed the output of one task into another, ensuring a seamless flow of information.

Expected outputs are defined to ensure the analyzed news articles are presented in a specific format. The output includes a breakdown of events, details, and significance, mirroring the desired structure for consistency and reliability.

The compilation task involves using context from previous tasks to create a final newsletter. Callback functions are introduced for this task, emphasizing the importance of building a chain of tasks from researching to analyzing and compiling news stories.

The code is refined by importing necessary modules like 'task' from 'crew AI' and 'datetime' to address missing references. This ensures the smooth execution of the news analysis process.

To set up the AI newsletter agents, import the AI newsletter agents from the agents file and the AI newsletter task from the task file. Next, set up the editor agent, news fetcher, and other agents as required.

For the fetch news task, pass in the news fetcher agent. For the analyze news task, pass in the news analyzer agent and the fetched news items as context. Remember to pass in an array for multiple context items. Lastly, for the compile newsletter task, pass in the news compiler agent, context from previous tasks, and a callback function.

Create a crew by passing in all the agents created earlier. Define tasks using the previously created tasks. Transition from sequential to hierarchical processing by setting the process to hierarchical. Define a manager llm to implement the hierarchical process.

To begin, import the crew and process modules from crew AI. Define the manager LLM to orchestrate the chain of command. Initialize an OpenAI GPT-4 instance for general use. Utilize Lang chain for this purpose.

Crew AI employs a hierarchical process that includes a pre-built manager agent for orchestrating tasks in a top-down fashion. This manager agent is specifically designed to oversee hierarchical operations within the crew.

After setting up the hierarchical process, a demonstration of the final output showcases the addition of the new process. This step provides a visual representation of how the hierarchical process integrates into the workflow.

Create a 'tools' directory and initialize it with an __init__.py file. Add a 'search_tools.py' file to implement search functionality. Use the tool decorator to enable the crew to recognize and utilize the search tool. The tool allows querying serper, a tool for internet searches, enhancing the crew's capabilities.

To use serper, one must create an API key by clicking 'reset key' and copying it. The key should be pasted in the ENV file with a specific name like 'serper API key'. This key allows serper to search the internet for news articles, providing titles, links, and dates of the top five results within the past 24 hours.

To resolve issues in the agents file, the search tools class from the tools folder needs to be imported. This step ensures that the agents can utilize the newly created search tools effectively.

To complete the setup, open AI keys must be obtained and environment variables updated. The secret key from open AI should be named 'open AI API key' and protected. Additionally, a callback function needs to be implemented to save the AI newsletter as a markdown file locally.

The process involves saving the final newsletter as a markdown file named with today's date, 'today's date.md'. The task output contains the results, which are written to the markdown file. The string content of the task output is obtained using 'task output.result'.

To fix missing dependencies, the code imports 'datetime' from the datetime module. This step ensures that the necessary dependencies are available for the script to run successfully.

The callback function is updated to use the 'save markdown' function. The 'save markdown' function is imported from the 'file.o' file. This update ensures that the markdown file is properly saved with the newsletter content.

The crew is initiated to start the process of generating the AI newsletter. The crew is kicked off using 'crew.kickoff'. Once the crew completes its task, the final results are printed using 'crew.work_results'.

Import errors related to misnaming files are identified and fixed. The 'AI newsletter' file is corrected to 'AI newsletter task' and 'agents' is updated to 'agent'. These corrections ensure that the script can access the necessary components.

To resolve the issue of not finding the OpenAI key, environment variables are loaded at the beginning of the 'main.py' file. This step ensures that the script can access and use the required environment variables throughout the project.

The AI article search tool is designed to search for new AI articles, extract titles, links, and generate snippets. It attempts to find dates and follows an expected format set up earlier. The tool runs autonomously, grabbing information and generating a Markdown file with today's date.

The AI newsletter was successfully generated on February 20th, 20244. It included recent information about a new AI development where an AI task force is being set up in the US to enforce regulations. The newsletter follows the format set up earlier and includes breaking news that occurred 6 hours ago.

The tutorial showcased the updated features of Crew AI, demonstrating how to build an AI newsletter. Viewers learned to utilize the new functionalities and create their own newsletters. The tutorial recommends exploring other AI tutorials on the channel and encourages viewers to continue learning and growing in the AI field.