AI-Powered Log Analysis System (Using ML)

Project Overview: "AI-Powered Log Analysis System" is an advanced security system that leverages Unsupervised Machine Learning to analyze massive system logs and detect anomalous behaviors and cyber threats in real-time.

How it Works: The system is powered by an Autoencoder AI model built with TensorFlow. It processes complex text logs (such as HDFS and Generic Server Logs), performs feature extraction to convert them into numerical vectors, and then uses the model to identify logs that deviate from the system's normal pattern, indicating a potential security threat or technical failure.

Key Features:

Interactive Dashboards: Includes a rapid-analysis interface built with Streamlit, and a custom, highly responsive dashboard designed with HTML/CSS/JS.

Robust Backend API: Developed using FastAPI to ensure high-speed processing of uploaded files and millisecond-latency delivery of analysis results to the frontend.

Big Data Processing: Utilizes Pandas and NumPy libraries for high-performance parsing and feature extraction of complex log data.

Real-time Threat Detection: Classifies logs based on an "Anomaly Score" and highlights critical keywords indicating attacks (e.g., SQL Injection attempts).

Technologies Used:

Python

TensorFlow / Keras

FastAPI

Streamlit

Pandas & NumPy

HTML, CSS, JavaScript