640 Course Project
0%

MSCS 640 — course portal

Network Intrusion Detection Challenge

Your mission: act as AI Security Engineers and build a Machine Learning-based Network Intrusion Detection System (NIDS). Your model must analyse raw network flow data and classify traffic as either “Benign” (normal) or a specific type of “Attack” (DDoS, port scan, botnet, and the rest) with the highest possible accuracy.

PDF
Contents
  1. •Project Overview
  2. •The Dataset
  3. •Core Objectives
  4. •Deliverables
  5. •The Competition
  6. •Rules of Engagement
  7. •Timeline
  8. •References

Project Overview

In modern Security Operations Centers (SOCs), analysts are overwhelmed by the sheer volume of network traffic logs. Traditional signature-based firewalls fail to catch zero-day attacks and sophisticated anomalies.

This project is a competition. All teams will compete on the same dataset. The teams with the highest performance metrics on the hidden test set will receive the highest marks (the “State-of-the-Art” Bonus).

The work runs as a pipeline. Raw CSE-CIC-IDS2018 flow CSVs go through three phases, and the Phase 2 models are then scored against a hidden test set to produce your leaderboard rank.

Build path

Raw CSVsPhase 1 — data engineeringPhase 2 — modelsPhase 3 — dashboard

Scoring path

Phase 2 — modelsHidden test setLeaderboard rank

The Dataset

We will use CSE-CIC-IDS2018, produced by the Canadian Institute for Cybersecurity. It captures ten days of traffic and ships the CICFlowMeter flow features as ML-ready CSVs.

  • Data source: the public, anonymously readable bucket https://cse-cic-ids2018.s3.amazonaws.com/, under Processed Traffic Data for ML Algorithms/. The instructor's whole-bucket mirror is the command below.
  • Where to start: develop and debug on the smallest day, Thursday-01-03-2018_TrafficForML_CICFlowMeter.csv (103 MB), then pull the remaining nine days once your pipeline runs end to end.
  • Features: 79 flow statistics per record — Flow Duration, Flow Byts/s, Flow Pkts/s, Tot Fwd Pkts, and the TCP flag counts, among others.
  • Target: multi-class classification on the Label column.
bash
aws s3 sync --no-sign-request s3://cse-cic-ids2018/ .
The largest day is about 4 GB, so plan for chunked reading rather than loading a whole file into memory at once.

Label values

Label spellings are taken verbatim from the CSVs:

  • Benign
  • FTP-BruteForce
  • SSH-Bruteforce
  • DoS attacks-GoldenEye
  • DoS attacks-Slowloris
  • DoS attacks-Hulk
  • DoS attacks-SlowHTTPTest
  • DDoS attacks-LOIC-HTTP
  • DDOS attack-HOIC
  • DDOS attack-LOIC-UDP
  • Brute Force -Web
  • Brute Force -XSS
  • SQL Injection
  • Infilteration
  • Bot
The misspelling Infilteration is the dataset's own, and the casing is inconsistent between DoS attacks- and DDOS attack-. Match the strings exactly as they appear in the files; do not “correct” them.

Each day carries Benign plus only the attacks staged that day, so build your training set across days rather than from one file.

Core Objectives

Phase 1: Data Engineering (The Foundation)

Real-world security data is dirty and imbalanced. You are required to:

1.
Clean the data: handle missing and infinite values, both common in network logs.
2.
Handle imbalance: network attacks are rare, roughly 1% of traffic, compared with normal traffic. You must apply techniques like SMOTE (Synthetic Minority Over-sampling Technique) or class weighting to ensure your model detects the attacks, not just the normal traffic.
3.
Feature selection: identify which network features — Total Fwd Packets, Flow Duration, and the like — are actually relevant to security.

Phase 2: Model Development (The Engine)

Develop a Deep Learning or Ensemble Machine Learning model.

  • Baseline: start with a Random Forest or XGBoost.
  • Advanced: implement a neural network — MLP, CNN-1D for sequence, or an autoencoder for anomaly detection.
  • Metric: we optimise for macro F1-score. Accuracy is misleading in imbalanced security datasets.

Phase 3: Deployment (The Product)

A model in a notebook is useless to a SOC analyst. You must build a web-based dashboard, using Streamlit or Flask, that:

1.
Accepts a CSV upload of new network traffic.
2.
Runs your pre-trained model.
3.
Displays a “Threat Report” highlighting which flows are malicious.

Deliverables

Each group must submit the following by the deadline:

1.
Git repository: must contain main.py, requirements.txt, and a README.md with instructions. Code must be clean and modular.
2.
Project paper (4 pages, IEEE format):
  • Abstract: summary of your approach.
  • Methodology: handling class imbalance and algorithm choice.
  • Results: confusion matrix, ROC curves, and F1-scores.
  • Conclusion: future improvements.
3.
Live demo or website: a URL, or a local demo, of your threat detection dashboard.

The Competition (Evaluation Criteria)

Grades are awarded based on a mix of technical rigour and competitive performance. Total points: 100.

ComponentPointsDescription
Data Pipeline15Quality of cleaning, EDA, and handling imbalance.
Methodology15Justification of model choice and hyperparameter tuning.
Reproducibility15Can the TA clone the repo and run it without errors?
The Website15UI/UX, functionality, and visualisation of results.
The Paper15Clarity, structure, and academic quality.
Presentation15Demo day delivery: the talk, the live demo, and your answers to questions.
Performance Rank10Based on the hidden test set leaderboard.
Bonus
  • 1st place: automatic 10 on performance, plus a 2% overall bonus.
  • 2nd place: automatic 10 on performance, plus a 1% overall bonus.
  • 3rd place and below: graded on a sliding scale relative to the baseline.

Rules of Engagement

⚠ Open source, not black boxes. You may use open-source libraries (Scikit-Learn, TensorFlow, PyTorch), but you cannot use a pre-trained “black box” API without explanation.

Collaboration: this is a team project of 2 members. All members must commit code to the Git repository.

AI use: you may use AI tools, but you must disclose how you used them in a short statement at the end of the paper. You are responsible for every claim: check each statement and each reference against its original source.

⚠ Citation integrity. Every cited work must exist and must support the sentence that cites it. A fabricated or misattributed reference is treated as academic misconduct.

Plagiarism: copying text from any source without quotation and citation is not accepted.

Timeline

Tick each milestone as you clear it; the ticks are remembered on this computer.

References

Iman Sharafaldin, Arash Habibi Lashkari and Ali A. Ghorbani. “Toward generating a new intrusion detection dataset and intrusion traffic characterization.” ICISSP, pages 108–116, 2018. https://www.unb.ca/cic/datasets/ids-2018.html