Resource Consumption in Network Traffic

Information about Resource Consumption in Network Traffic

Published on June 15, 2007

Author: Abbott

Source: authorstream.com

Content

Automatically Inferring Patterns of Resource Consumption in Network Traffic:  Automatically Inferring Patterns of Resource Consumption in Network Traffic Cristian Estan, Stefan Savage, George Varghese University of California, San Diego Who is using my link?:  Who is using my link? Looking at the traffic:  Do something smarter! Too much data for a human Looking at the traffic Looking at traffic aggregates:  Looking at traffic aggregates Aggregating on individual packet header fields gives useful results but Traffic reports are not always at the right granularity (e.g. individual IP address, subnet, etc.) Cannot show aggregates defined over multiple fields (e.g. which network uses which application) The traffic analysis tool should automatically find aggregates over the right fields at the right granularity Most traffic goes to the dorms … What apps are used? Where does the traffic come from? …… Which network uses web and which one kazaa? Ideal traffic report:  Ideal traffic report Web is the dominant application The library is a heavy user of web That’s a big flash crowd! This is a Denial of Service attack !! This paper is about giving the network administrator insightful traffic reports Contributions of this paper:  Contributions of this paper Approach Definitions Algorithms System Experience Approach:  Approach Characterize traffic mix by describing all important traffic aggregates Multidimensional aggregates (e.g. flash crowd described by protocol, port number and IP address) Aggregates at the the right level of granularity (e.g. computer, subnet, ISP) Traffic analysis is automated – finds insightful data without human guidance Definition: traffic clusters:  Definition: traffic clusters Traffic clusters are the multidimensional traffic aggregates identified by our reports A cluster is defined by a range for each field The ranges are from natural hierarchies (e.g. IP prefix hierarchy) – meaningful aggregates Example Traffic aggregate: incoming web traffic for CS Dept. Traffic cluster: ( SrcIP=*, DestIP in 132.239.64.0/21, Proto=TCP, SrcPort=80, DestPort in [1024,65535] ) Definition: traffic report:  Traffic reports give the volume of chosen traffic clusters To keep report size manageable describe only clusters above threshold (e.g. H=total of traffic/20) To avoid redundant data compress by omitting clusters whose traffic can be inferred (up to error H) from non-overlapping more specific clusters in the report To highlight non-obvious aggregates prioritize by using unexpectedness label Example 50% of all traffic is web Prefix B receives 20% of all traffic The web traffic received by prefix B is 15% instead of 50%*20%=10%, unexpectedness label is 15%/10%=150% Definition: traffic report Contributions of this paper:  Contributions of this paper Approach Definitions Algorithms System Experience Algorithms and theory:  Algorithms and theory Algorithms and theoretical bounds in the paper Unidimensional reports are easy to compute Multidimensional reports are exponentially harder as we add more fields Next few slides Example of unidimensional compression Example for the structure of the multidimensional cluster space Unidimensional report example:  Unidimensional report example 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.8 10.0.0.9 10.0.0.10 10.0.0.14 15 35 30 40 160 110 35 75 Hierarchy Threshold=100 10.0.0.14/31 10.0.0.12/30 Unidimensional report example:  Unidimensional report example 10.0.0.8 10.0.0.9 10.0.0.0/29 10.0.0.8/29 120 380 160 110 Compression 305-270andlt;100 380-270≥100 Multidimensional structure ex.:  Multidimensional structure ex. Nodes (clusters) have multiple parents US Web Nodes (clusters) overlap CA Contributions of this paper:  Contributions of this paper Approach Definitions Algorithms System Experience System: AutoFocus:  System: AutoFocus Traffic parser Web based GUI Cluster miner Grapher Packet header trace Slide17:  Slide18:  Slide19:  Contributions of this paper:  Contributions of this paper Approach Definitions Algorithms System Experience Structure of regular traffic mix:  Backups from CAIDA to tape server Semi-regular time pattern FTP from SLAC Stanford Scripps web traffic Web andamp; Squid servers Large ssh traffic Steady ICMP probing from CAIDA Structure of regular traffic mix SD-NAP SD-NAP Analysis of unusual events:  Analysis of unusual events UCSD to UCLA route change Sapphire/SQL Slammer worm Site 2 Conclusions:  Conclusions 1010111101010000101011111101011001010101101011010000101010100101010111101010101000101111010000010111111101011001010111010111100100101010100011011111100010101110110101100101010110101111000010101011110111010111010101010111111010110010101011010101111101010000110100001011010100101011001000000101011001010101011111000010001000010101011110101000010111001010101101011110000010101011111101011000101111010000010111110101011010111100100101010110010101010001010100101010110101010010111001010000010100001110110101010110111111000101011101011101011001010101101011110000110111101110101110101010101111110101100101010110101111011101010000110101010010101101010111010101001010000101011010101001010100000101010101010101101011101010100000010101010101101010101011110101110101011010100011000101010010111010101001101010100001000110101111010100010110 Conclusions:  Conclusions Multidimensional traffic clusters using natural hierarchies describe traffic aggregates Traffic reports using thresholding identify automatically conspicuous resource consumption at the right granularity Compression produces compact traffic reports and unexpectedness labels highlight non-obvious aggregates Our prototype system, AutoFocus, provides insights into the structure of regular traffic and unexpected events Thank you!:  Thank you! Alpha version of AutoFocus downloadable from http://ial.ucsd.edu/AutoFocus/ Any questions? Acknowledgements: NIST, NSF, Vern Paxson, David Moore, Liliana Estan, Jennifer Rexford, Alex Snoeren, Geoff Voelker Bounds and running times:  Bounds and running times Open questions:  Open questions Are there tighter bounds for the size of the reports? Are there algorithms that produce smaller results? Are there algorithms that compute traffic reports more efficiently? In streaming fashion? Delta reports:  Delta reports Why repeat the same traffic report if the traffic doesn’t change from one day to the other? Delta reports describe the clusters that increased or decreased by more than the threshold from one interval to the other On related traffic mixes delta reports much smaller than traffic reports Multidimensional compression very hard for delta reports We have only exponential algorithm for the cluster delta Greedy compression algorithm:  Greedy compression algorithm Multidimensional report example:  Multidimensional report example Thresholding Compression System details:  System details

Related presentations


Other presentations created by Abbott

Fantastic Pictures
31. 07. 2007
0 views

Fantastic Pictures

jovanovi
02. 05. 2008
0 views

jovanovi

ours2
30. 04. 2008
0 views

ours2

golding
28. 04. 2008
0 views

golding

SandersD
22. 04. 2008
0 views

SandersD

MasterclassBasicsOfH edging
18. 04. 2008
0 views

MasterclassBasicsOfH edging

Energy Management Basics
17. 04. 2008
0 views

Energy Management Basics

jpn ma
13. 04. 2008
0 views

jpn ma

0802 Roderick Deane
10. 04. 2008
0 views

0802 Roderick Deane

Athirapally Vazhachaal Falls
26. 09. 2007
0 views

Athirapally Vazhachaal Falls

Lecture7
10. 10. 2007
0 views

Lecture7

qa overview cmu
16. 10. 2007
0 views

qa overview cmu

11 GUIDE presentation Porvoo 10
19. 10. 2007
0 views

11 GUIDE presentation Porvoo 10

Egyptian Quiz
21. 10. 2007
0 views

Egyptian Quiz

AGE200524122005
24. 10. 2007
0 views

AGE200524122005

Henry VIII wwt bam
21. 08. 2007
0 views

Henry VIII wwt bam

DOE LNG FORUM 14
07. 11. 2007
0 views

DOE LNG FORUM 14

matzgdh6
22. 10. 2007
0 views

matzgdh6

BdReviewFINAL2006 lq
16. 11. 2007
0 views

BdReviewFINAL2006 lq

session2
20. 11. 2007
0 views

session2

lecture26 fall 2003
19. 11. 2007
0 views

lecture26 fall 2003

WWF
26. 11. 2007
0 views

WWF

Chocolate 01
03. 10. 2007
0 views

Chocolate 01

MARE 494 Lecture 12
12. 10. 2007
0 views

MARE 494 Lecture 12

ErnestoRivera
22. 10. 2007
0 views

ErnestoRivera

InherentSafety Westray
10. 12. 2007
0 views

InherentSafety Westray

tema1
24. 10. 2007
0 views

tema1

Libecap powerpoint
04. 01. 2008
0 views

Libecap powerpoint

Assam
05. 01. 2008
0 views

Assam

Samuels Mol Pharm 11 03
24. 02. 2008
0 views

Samuels Mol Pharm 11 03

Stock Market Game
24. 02. 2008
0 views

Stock Market Game

nv specialitypp
29. 10. 2007
0 views

nv specialitypp

104 04MagnConst
13. 11. 2007
0 views

104 04MagnConst

clingan
04. 03. 2008
0 views

clingan

EARNIN x007E 1 2
23. 10. 2007
0 views

EARNIN x007E 1 2

aes
13. 03. 2008
0 views

aes

Semprini
16. 03. 2008
0 views

Semprini

KPMG PresentationSMEfinal
26. 03. 2008
0 views

KPMG PresentationSMEfinal

temple of Zeus
21. 08. 2007
0 views

temple of Zeus

Australia 2 March 20051
28. 09. 2007
0 views

Australia 2 March 20051

maillistnews
05. 10. 2007
0 views

maillistnews

feb 5
27. 09. 2007
0 views

feb 5

ykis06 K F Liu
15. 11. 2007
0 views

ykis06 K F Liu

schreiber
12. 10. 2007
0 views

schreiber

Pisani
30. 10. 2007
0 views

Pisani

200752414331362
11. 10. 2007
0 views

200752414331362

decisiontheoryreview
04. 01. 2008
0 views

decisiontheoryreview

a Macroecologia Alimentar 2
22. 10. 2007
0 views

a Macroecologia Alimentar 2

praesentationschule
02. 11. 2007
0 views

praesentationschule

Wireless Data Business
18. 06. 2007
0 views

Wireless Data Business

WHA userguide Customer2006
18. 06. 2007
0 views

WHA userguide Customer2006

wales
18. 06. 2007
0 views

wales

VON04s I2
18. 06. 2007
0 views

VON04s I2

UTeV Rick Field 12 14 06
18. 06. 2007
0 views

UTeV Rick Field 12 14 06

UDDI Overview Presentation
18. 06. 2007
0 views

UDDI Overview Presentation

2006 02 28T124652
25. 03. 2008
0 views

2006 02 28T124652

SoprolecheAgo 2005
23. 10. 2007
0 views

SoprolecheAgo 2005

ZadehTalk
18. 10. 2007
0 views

ZadehTalk

ihepccc hepix
24. 10. 2007
0 views

ihepccc hepix

jedwards
02. 10. 2007
0 views

jedwards

Wireless Mobile Platform OAEP
15. 06. 2007
0 views

Wireless Mobile Platform OAEP

Taverna Workbench
15. 06. 2007
0 views

Taverna Workbench

Job Services with Genius Portal
15. 06. 2007
0 views

Job Services with Genius Portal

MSN screenshots
15. 06. 2007
0 views

MSN screenshots

Caltech, Peck Fest Jan 05
15. 06. 2007
0 views

Caltech, Peck Fest Jan 05

rslt3218
17. 10. 2007
0 views

rslt3218

6 26
17. 10. 2007
0 views

6 26

vp sigmetrics 04 adversaries
18. 06. 2007
0 views

vp sigmetrics 04 adversaries

calcagno
22. 10. 2007
0 views

calcagno

LegoDesign
31. 12. 2007
0 views

LegoDesign

Ted Hanss Internet2
15. 06. 2007
0 views

Ted Hanss Internet2

Mary and Darnley
21. 08. 2007
0 views

Mary and Darnley

1 partie
17. 10. 2007
0 views

1 partie

Techmission Safe Families
15. 06. 2007
0 views

Techmission Safe Families

Camille Page CrocodileGuy MK8
11. 10. 2007
0 views

Camille Page CrocodileGuy MK8

Salandayia
04. 10. 2007
0 views

Salandayia

B4 Hamilton
03. 01. 2008
0 views

B4 Hamilton

SEC062804
25. 10. 2007
0 views

SEC062804