國立台灣科技大學 資訊工程系所

智慧型系統實驗室


研 究 資 源


Information Security

Intrusion Detection System


Tutorial/Survey

1. Aleksandar Lazarevic, Vipin Kumar, Jaideep Srivastava, "DATA MINING   FOR INTRUSION DETECTION," Tutorial on the Pacific-Asia Conference on knowledge Discovery in Databases 2003.

2. Anita K. Jones and Robert S. Sielken, "Computer System Intrusion Detection:A Survey1," University of  Virginia Computer Science Department 1999.

3. Aleksandar Lazarevic, Vipin Kumar, Jaideep Srivastava, "Intrusion Detection: a  Survey, Chapter 2," in book: "Managing Cyber Threats: Issues, Approaches and  Challenges" Series: Massive Computing, Vol. 5,      Vipin Kumar, Jaideep  Srivastava,  Aleksandar Lazarevic (Eds.) 2005, Springer Publisher.

4. Rodolfo Villarroela, Eduardo Fernandez-Medinab, Mario Piattinib, "Secure   information systems  development e a survey and comparison," Computers & Security (2005) 24, 308-321.

5. D. Engelhardt, "Directions for Intrusion Detection and Response: A survey,"     DSTO   Electronics and  Surveillance Research Laboratory, Department of    Defense, Australia Technical Report DSTO-GD-0155,
1997.

6. S TERRY BRUGGER, "Data Mining Methods for Network Intrusion    Detection,"  ACM Journal 2004


Data Sets

DARPA 1998 data set
KDDCup99 data set (DARPA 1998 modification)
DARPA 1999 data set
System call traces data set2 – U. New Mexico

Lincoln Laboratory :
Solaris audit data using BSM3 (Basic Security Module)

University of Melbourne, Australia MOAT – packet trace files Auckland II – packet trace files
 1. http://pma.nlanr.net/Traces/
 2. http://pma.nlanr.net/Traces/Traces Auckland II – packet trace files

Data set with virus files4 available from Columbia University Data sets in Intrusion Detection

1. http://www.ll.mit.edu/IST/ideval/data/data_index.html
2. http://www.cs.unm.edu/~immsec/systemcalls.htm
3. Sun Microsystems SunShield Basic Security Module Guide.
4. http://www.cs.columbia.edu/ids/mef/software


2000 DARPA Dataset 簡介

2000 DARPA Intrusion Detection Scenario Specific Data Sets

離線入侵偵測資料集的生產是經由在 Wisconsin Re-Think 和 July 2000 Hawaii PI 會議共同完成.

主要有 2 劇本的攻擊方式

LLDOS 1.0 - Scenario One

第一個劇本包含了分散的阻絕服務(distributed denial of service)攻 擊,此攻擊實作在許多部分的網路環境和審核連線。在網路攻擊者的探測,這些連線把 5 個階段攻擊分組,根據 Solaris sadmind 的 弱點入侵主機,安裝 trojan DDOS 軟體,然後展開 DDOS 攻擊。

LLDOS 2.0.2 - Scenario Two

第二個劇本大致與第一個劇本想同,但加入一些祕密的(stealthy)的攻擊。


參考資料

Lincoln Laboratory Publications 2001

Joshua Haines, Lee Rossey, Rich Lippmann and Robert Cunnigham, "Extending the 1999 Evaluation", In the Proceedings of DISCEX 2001, June 11-12, Anaheim, CA. [PDF]

Joshua W. Haines, Richard P. Lippmann, David J. Fried, Eushiuan Tran, Steve Boswell, Marc A. Zissman, "1999 DARPA Intrusion Detection System Evaluation: Design and Procedures", MIT Lincoln Laboratory Technical Report, [PDF]

Lincoln Laboratory Publications 2000

Richard Lippmann, Joshua W. Haines, David J. Fried, Jonathan Korba, Kumar Das "The 1999 DARPA Off-Line Intrusion Detection Evaluation", Draft of paper submitted to Computer Networks, In Press, 2000. [PDF]

Jonathan Korba, "Windows NT Attacks for the Evaluation of Intrusion Detection Systems", S.M. Thesis, Massachusetts Institute of Technology, June, 2000. [PDF]

Richard P. Lippmann, David J. Fried, Isaac Graf, Joshua W. Haines,Kristopher R. Kendall, David McClung, Dan Weber, Seth E. Webster, Dan Wyschogrod, Robert K. Cunningham, and Marc A. Zissman, "Evaluating Intrusion Detection Systems: the 1998 DARPA Off-Line Intrusion Detection Evaluation", Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000, Vol. 2, pp. [PDF]

Related Publications 2000

T.Bowen, D.Chee, M.Segal, R. Sekar, T. Shanbhag, P. Uppuluri, "Building Survivable Systems: An Integrated Approach based on Intrusion Detection and Damage Containment," Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000.

Salvatore J. Stolfo, Wei Fan, Wenke Lee,
"Cost-basedModeling for Fraud and Intrusion Detection:Results from the JAM Project",
Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000.

Giovanni Vigna,Steve T. Eckmann, Richard A. Kemmerer, "The STAT Tool Suite",
Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000.

John McHugh, "The Lincoln Laboratory Intrusion Detection Evaluation: A Critique",
Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000.

Brad J. Wood, Ruth A. Duggan, "Red-Teaming of Advanced Information Assurance Concepts", Proceedings of the 2000 DARPA Information Survivability Conference and Exposition, 2000.

 

網 路環境分為 inside ,outside,dmz 三個網路區段。

       

Outside Hosts

IP Address

Hostname

Operating System

Notes

135.13.216.191

alpha.apple.edu

Linux Redhat 5.0

kernel 2.0.32

135.8.60.182

beta.banana.edu

Solaris 2.5.1

 

194.27.251.21

gamma.grape.mil

SunOS 4.1.4

 

194.7.248.153

delta.peach.mil

Linux Redhat 5.0

kernel 2.0.32

195.115.218.108

epsilon.pear.com

Solaris 2.5.1

 

195.73.151.50

lambda.orange.com

SunOS 4.1.4

 

196.37.75.158

jupiter.cherry.org

Linux Redhat 5.0

kernel 2.0.32

196.227.33.189

saturn.kiwi.org

Solaris 2.5.1

 

197.182.91.233

mars.avocado.net

SunOS 4.1.4

 

197.218.177.69

pluto.plum.net

Linux Redhat 5.0

kernel 2.0.32

192.168.1.30

monitor.af.mil

MacOS

AF SNMP monitor

192.168.1.10

calvin.world.net

 

Outside gateway

192.168.1.20

aesop.world.net

 

Outside Web Server

192.168.1.1

loud.world.net

 

Cisco 2514 Router


DMZ Hosts

IP Address

Hostname

Operating System

Notes

172.16.114.1

loud.eyrie.af.mil

 

Cisco 2514 Router

172.16.114.2

firewall.eyrie.af.mil

 

Sidewinder Firewall

172.16.114.10

plato.eyrie.af.mil

Solaris 2.6

Simulation Coordinator

172.16.114.20

smith.eyrie.af.mil

Solaris 2.7

Loghost -- not used

172.16.114.30

solomon.eyrie.af.mil

Solaris 2.7

DMZ Sniffer

172.16.114.50

marx.eyrie.af.mil

Linux Redhat 4.2

kernel 2.0.27


Inside Hosts

IP Address

Hostname

Operating System

Notes

172.16.115.1

firewall-inside.eyrie.af.mil

 

Inside Firewall Interface

172.16.116.1

firewall-inside.eyrie.af.mil

 

Inside Firewall Interface

172.16.117.1

firewall-inside.eyrie.af.mil

 

Inside Firewall Interface

172.16.118.1

firewall-inside.eyrie.af.mil

 

Inside Firewall Interface

172.16.112.10

locke.eyrie.af.mil

Solaris 2.6

Inside Sniffer

172.16.112.20

hobbes.eyrie.af.mil

Linux Redhat 5.0

Inside gateway, kernel 2.0.32

172.16.112.50

pascal.eyrie.af.mil

Solaris 2.5.1

 

172.16.112.100

hume.eyrie.af.mil

Windows NT 4.0

Build 1381, Service Pack 1

172.16.112.149

eagle.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.112.194

falcon.eyrie.af.mil

Solaris 2.5.1

 

172.16.112.207

robin.eyrie.af.mil

SunOS 4.1.4

 

172.16.113.50

zeno.eyrie.af.mil

SunOS 4.1.4

 

172.16.113.84

duck.eyrie.af.mil

SunOS 4.1.4

 

172.16.113.105

swallow.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.113.204

goose.eyrie.af.mil

Solaris 2.5.1

 

172.16.113.148

crow.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.113.168

finch.eyrie.af.mil

SunOS 4.1.4

 

172.16.113.169

swan.eyrie.af.mil

Solaris 2.5.1

 

172.16.113.207

pigeon.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.115.5

pc1.eyrie.af.mil

Windows 95

 

172.16.115.20

mill.eyrie.af.mil

Solaris 2.7

Eyrie AFB DNS server

172.16.115.87

pc2.eyrie.af.mil

Windows 95

 

172.16.115.234

pc0.eyrie.af.mil

Window NT 4.0

Build 1381, Service Pack 1

172.16.116.44

pc5.eyrie.af.mil

Windows 3.1

 

172.16.116.194

pc3.eyrie.af.mil

Windows 95

 

172.16.116.201

pc4.eyrie.af.mil

Windows 95

 

172.16.117.52

pc7.eyrie.af.mil

Windows 3.1

 

172.16.117.103

pc9.eyrie.af.mil

MacOS

 

172.16.117.111

pc8.eyrie.af.mil

MacOS

 

172.16.117.132

pc6.eyrie.af.mil

Windows 3.1

 

172.16.118.10

linux1.eyrie.af.mil

Linux Redhat 5.2

kernel 2.0.36

172.16.118.20

linux2.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.30

linux3.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.40

linux4.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.50

linux5.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.60

linux6.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.70

linux7.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.80

linux8.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.90

linux9.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

172.16.118.100

linux10.eyrie.af.mil

Linux Redhat 5.0

kernel 2.0.32

 

Reference Books

1.David J. Marchette
"Computer Intrusion Detection and Network Monitoring A Statistical Viewpoint"
Springer 2001

2.Stephen Northcutt, Lenny Zeltser, Scott Winters, Karen Fredrick, Ronald W. Ritchey
"Inside Network Perimeter Security: The Definitive Guide to Firewalls, Virtual Private Networks (VPNs), Routers, and Intrusion Detection Systems" 


3.Stephen Northcutt, Judy Novak
"Network Intrusion Detection (3rd Edition) " 

Alert Correlation


Alert Correlation Introduction

‧    Due to the limitation of IDSs, false positives are big problem.

–    Up to 99% of alerts are false positives!

‧    [Julisch 2001]

–    Reason:

‧    Runtime limitation, dependency on environment, specificity of detection signatures, etc.

–    Manual investigation is labor intensive and error prone

Goal of Alert Correlation

‧    Input: alerts generated from multiple IDSs

‧    Output: a succinct, high-level view of the attacks (filtering out incorrect alerts)

‧    Open problems:

–    Alert Fusion

–    False Alarm Reduction

–    Multi-step correlation

–    Alert Verification

‧    Algorithms

–    Knowledge based (require the topology of network)

–    Data mining based

Alert Fusion

‧    Combine alerts referring to the same event, reported from different IDSs.

–    Not easy when link network-based and host-based alerts

‧    Fuse alerts to create a higher level view or scenario of the attacks

–    [Dain and Cunningham 2001], Fusing alerts into scenarios.

–    Each scenario indicates a sequence of actions performed by a single actor (not necessarily attacks)

‧    Ex. Port scan ->Apache buffer overflow exploit ->local exploit      ->elevates his/her privileges

–    Heuristically add a new alert to existing scenarios by comparing it with the most recent alerts in each scenarios

–    Data: hand tagging scenarios of DEF CON data

False Positive Reduction

‧    Root cause analysis [Julisch 2003]

–    Detect large clusters of alerts

–    Alerts in a cluster are summarized to a general alert

–    Assuming each cluster has a root cause

‧    Ex. A broken TCP/IP stack can triggers large amounts of “fragmented IP” alarms with the same source IP address, and the same source port

–    Leave the work for discovering what root causes are to human being -> semiautomatic

–    Removing these root causes, the future alarms could be reduced 87%

‧    Association rules [Manganaries 2000]

–    Assuming frequent behavior, over extended periods of time, is likely to be normal

–    A combination of frequent alarm types always in the same order is normal

–    Alarms that are consistent with these association rules are deemed normal and discarded

‧    Episode mining [Clifton and Gengo 2000]

–    Making the same assumption with Manganaries

–    Using episode mining to find frequent sequential items

‧    Multi-step correlation

–    Identify high-level attack patterns composed of several individual attacks

–    Most works contain defining post-condition and pre-condition of an attack, and match them to get the logic links

‧    [Cuppens 2002]

‧    Alert verification  

–    Required to know the topology of the target network

–    Using vulnerability dependency information and attack characteristics

‧    Ex. A specific attack can cause what kind of problem in the net work

Reference

‧    [Valeur, 2004] A comprehensive approach to intrusion detection alert correlation

‧    [Manganaris, 2000] A data mining analysis of RTID alarms

‧    [Dain and Cunningham, 2001] Fusing heterogeneous alert stream into scenarios

‧    [Zhu and Ghorbani, 2005] Alert Correlation for extracting attack strategies

‧    [Clifton and Gengo, 2000] Developing custom intrusion detection filters using data mining

‧    [Valdes and Skinner, 2001] Probabilistic alert correlation

‧    [Julisch, 2003] Clustering intrusion detection alarms to support root cause analysis

‧    [Pietraszek 2004] Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection

Anti-Spam


1.Xiao-Lin Wang and Cloete, I. "Learning to Classify Email: A Survey"
in Machine Learning and Cybernetics, 2005. Proceedings of 2005
International Conference on Volume 9,? 18-21 Aug. 2005 Page(s)
2.Maria Gomez Hidalgo "Machine Learning for Spam Detection References" January 31, 2005"


Tutorial/Survey

1.
Xiao-Lin Wang and Cloete, I.
”Learning to Classify Email: A Survey”
in Machine Learning and Cybernetics, 2005. Proceedings of 2005 International Conference on Volume 9,? 18-21 Aug. 2005 Page(s):5716 – 5719

2.
Jose Maria Gomez Hidalgo ”Machine Learning for Spam Detection References”  , January 31, 2005


Paper Collection

1. Jennifer Golbeck and James Hendler.; ”Reputation network analysis for email filtering.” In Proceedings of the First Conference on Email and Anti-Spam (CEAS), 2004. Available: http://www.ceas.cc/papers-2004/177.pdf

2. G. Sakkis, I. Androutsopoulos, G. Paliouras, V. Karkaletsis, C. D. Spyropoulos, and P. Stamatopoulos.; ”A memory-based approach to anti-spam filtering for mailing lists.” Information Retrieval Journal, 6(1), 2003

3. P. Boykin and V. P. Roychowdhury; ”Leveraging social networks to fight
spam. ” Computer, vol. 38, no. 4, pp. 61–68, April 2005.

4. Harris Drucker, Vladimir Vapnink, and Dongui Wu.; ”Support vector machines for spam categorization. ” IEEE Tansactions on Neural Networks, 10(5):1048—1054, 1999.

5. V Zorkadis, DA Karras, M Panayotou ; ”Efficient information theoretic strategies for classifier combination, feature extraction and performance evaluation in improving false positives and false negatives for spam e-mail filtering”. Neural Networks, Volume 18 , Issue 5-6 (June 2005) , 2005 Special issue,Pages: 799 - 807 .


Data Sets

1. G. Cormack, T. Lynam, University of Waterloo; ”TREC 2005 Spam Track Overview” TREC 2005 - The Fourteenth TExt Retrieval Conference.

2. B. Klimt and Y. Yang, “The enron corpus: A new dataset for email classification research.” In ECML, 2004, pp. 217–226.


Reference Books

1. Paul Wolfe, Charlie Scott?and?Mike Erwin; ”Anti-Spam Toolkit” McGraw-Hill/Osborne c 2004.

2. Jonathan A. Zdziarski; ”Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification” No Starch Press 2005.