Event Logs Management InfoSecHK

Information about Event Logs Management InfoSecHK

Published on January 21, 2008

Author: Taddeo

Source: authorstream.com

Content

Event Logs Management:  Event Logs Management Anthony LAI, CISSP, CISA Founder & Editor InfoSec Hong Kong Why do I come up an idea to present this?:  Why do I come up an idea to present this? First of all, I am not the event log expert but the one suffered from event log burden.  Think of ideas to improve it in a bank. No budget and complicated approval process there , forcing me to seek for free software. InfoSec Hong Kong www.infosechk.org:  InfoSec Hong Kong www.infosechk.org Goal Promote security awareness to public and education sectors. Our advantages We act as a bridge between security concepts and commercial solutions Members Over 370 registered members Enjoy discounted products from our sponsors Current Status Over 64,000 visit since Aug 2004 News, seminar, training, advertisement, consultancy service for SMEs. Future Reflect the voice to government from public and focus on more security education and provide solutions to public. It is a long term commitment Invitation for more commercial sponsorship Author Profile:  Author Profile Anthony Lai a speaker for the International Information Systems Security Certification Consortium, Inc. [(ISC)2®], the non-profit international leader in educating, qualifying and certifying information security professionals worldwide. (ISC)2 issues the Certified Information Systems Security Professional (CISSP) and Systems Security Certified Practitioner (SSCP) credentials and related concentrations to those meeting necessary competency requirements. Anthony is also a founder and editor of the “InfoSec Hong Kong” Website with over 300 registered public members. He currently is a chapter leader of OWASP (The Open Web Application Security Project- HK Chapter) and serves as a program committee member in PISA (Professional Information Security Association) and ISSA (HK Chapter). His major interest is in web application security, various penetration tests and forensics. He is a column writer on information security in Hong Kong local newspaper – “Apple Daily”. He has certified as CISSP and CISA. What will I cover?:  What will I cover? Event Logs Monitoring Problems Solutions (You may most concern about it) Demonstration (You may most be interested in it ) Experience sharing among all of you. Real-Case: Event Log Monitoring Problem in a Bank:  Real-Case: Event Log Monitoring Problem in a Bank It have 50 systems, and 12 windows servers and other legacy systems. In the past, the one taking the information security officer just use his “X-Ray” Eyes to scan over all the Logs print-outs (over 400~600 papers) to detect errors every working day (Remark: I believe he is readily a superman, otherwise, he will be blind!) No filtering, No selection for critical events. Use half day to review the logs. No standard document about log review process. No solutions from other branches in other countries. Real-Case: Event Log Monitoring Problem in a Bank:  Real-Case: Event Log Monitoring Problem in a Bank Their current solutions: Print out the event logs with heaps of papers, and glance over it and then chop, chop chop with their signature. Package them up and put into the inventory Auditor has no comment on this kind of log review process. The Challenges:  The Challenges Bureaucratic approval process ( I can’t count how many tiers I need to pass even I just want to get a few thousand dollars) Free and cost saving Efficient to solve the problem Shorten the log review time Automate the log download process Patch the log file with appropriate headers and format. Save our world, save papers! In addition, increase the opportunity to detect any critical events and suspicious activity. No one has knowledge in this area. What did I suggest to bank?:  What did I suggest to bank? Long Term Security Operation Center (SOC) (but I think it is too early because I cannot find they have put some efforts in Incidence Response Planning and Implementation) Logs Filtering and Processing Server Short & Mid Term Commercial event log review software Roadmap to building log processing standard:  Roadmap to building log processing standard Log Nature and Element Define the elements needs for a log records What are the critical events? Operation Log patching, download, process and retention as well as purging. Review What are we looking at? The procedure to report any suspicious events. More challenges from my observation:  More challenges from my observation Various systems with various standard in logging -> Hard to integrate. Cultural problem and political problem No one thinks about the system infrastructure. Slide13:  Log Wiping Risk and Attack Event Logs Monitoring Tools:  Event Logs Monitoring Tools Microsoft Log Parser 2.2 http://www.microsoft.com/technet/scriptcenter/tools/logparser/default.mspx Kiwi Syslog Products: http://www.kiwisyslog.com/Syslogs Remstats - http://remstats.sourceforge.net/release/log-server.html Set up a Linux log server http://www.linuxsecurity.com/content/view/117514/49/ Log Parser:  Log Parser Search for Data - Search for the logons of a specific user among the events in the Windows Event Log: C:\>LogParser "SELECT TimeGenerated, SourceName, EventCategoryName, Message INTO report.txt FROM Security WHERE EventID = 528 AND SID LIKE '%TESTUSER%'" -resolveSIDs:ON Log Parser (2):  Log Parser (2) Create Reports - Create custom-formatted HTML reports. Log Parser (3):  Log Parser (3) Calculate Statistics -Calculate the distribution of the HTTP response status codes from your IIS log files: C:\>LogParser "SELECT sc-status, COUNT(*) AS Times INTO Chart.gif FROM <1> GROUP BY sc-status ORDER BY Times DESC" – chartType:PieExploded3D -chartTitle:"Status Codes" And produce a chart formatted as desired: Log Parser - Syntax:  Log Parser - Syntax Examples: LogParser "SELECT date, REVERSEDNS(c-ip) AS Client, COUNT(*) FROM file.log WHERE sc-status<>200 GROUP BY date, Client" -e:10 LogParser file:myQuery.sql?myInput=C:\temp\ex*.log+myOutput=results.csv LogParser -c -i:BIN -o:W3C file1.log file2.log "ComputerName IS NOT NULL" Help: -h GRAMMAR : SQL Language Grammar -h FUNCTIONS [ <function> ] : Functions Syntax -h EXAMPLES : Example queries and commands -h -i:<input_format> : Help on <input_format> -h -o:<output_format> : Help on <output_format> -h -c : Conversion help Log Parser – Sample Output:  Log Parser – Sample Output Server EventID Total ------- ------ ----- HKGKABS1 528 420 HKGKABS1 529 1 HKGKABS1 538 419 HKGKABS1 539 1 HKGKABS1 576 420 HKGKABS1 578 2 HKGUATS1 528 73 HKGUATS1 538 71 HKGUATS1 576 73 HKGUATS1 578 11  …………. Statistics: ----------- Elements processed: 1130 Elements output: 10 Execution time: 0.19 seconds From logs download to output reports (1a):  From logs download to output reports (1a) Automatic Download strTarget = ""“P:\SecurityLogs\log\t_Server1.txt""" Set objShell = CreateObject("WScript.Shell") Set objExec = objShell.Exec("cmd.exe /C dumpel -f " & strTarget & " -s Server1 -d 1 -l security -c -format dtIucs") strPingResults = LCase(objExec.StdOut.ReadAll) ‘dumpel is an executable to download logs From logs download to output reports (1b):  From logs download to output reports (1b) DUMPEL Usage: dumpel -f file [-s \\server] [-l log [-m source]] [-e n1 n2 n3..] [-r] [-t] [-d x] -d <days> Filters for event last days (number larger than zero) -e nn Filters for event id nn (up to 10 may be specified) -f <filename> Output filename (default stdout) -l <name> Dumps the specified log (system, application, security) -b Dumps a backup file (use -l to specify file name) -m <name> Filters for events logged by name -r Filters out events logged by name (must use -m too) -s <servername> Remote to servername -t Use tab to separate strings (default is space) -c Use comma to separate fields -ns Do not output strings -format <fmt> Specify output format. Default format is dtTCISucs where t - time d - date T - event type C - event category I - event ID S - event source u - user c - computer s - strings From logs download to output reports (2a):  From logs download to output reports (2a) strTarget1 = ""“Q:\log\window\script\Daily\header.txt""" strTarget2 = ""“P:\SecurityLogs\log\" & genmonth & genday & "t_Server1.txt""" ‘Copy log files to target directory with appended header Set objShell = CreateObject("WScript.Shell") Set objExec = objShell.Exec("cmd.exe /C copy " & strTarget1 & "+" & strTarget & " " & strTarget2) strPingResults = LCase(objExec.StdOut.ReadAll) `Invoke and run Log Parser, it outputs chart. Set objShell = CreateObject("WScript.Shell") Set objExec = objShell.Exec("cmd.exe /C P:\SecurityLogs\Logparser.exe file:P:\SecurityLogs\EventIDDistrib_Overview.sql?sourcefile=P:\SecurityLogs\2005log\" & genmonth & genday & "t_Server1.txt+destfile=P:\SecurityLogs\BarChart\" & genmonth & genday & “Overview_Server1.gif -i:csv -o:CHART -charttype:BarClustered -categories:ON -values:ON -charttitle:Event_Logs_Overview_Report_" & genmonth & genday & “Server1 -legend:ON -groupsize:1024x700") strPingResults = LCase(objExec.StdOut.ReadAll) From logs download to output reports (2b):  From logs download to output reports (2b) ‘Output CSV file with logs breakdown Set objShell = CreateObject("WScript.Shell") Set objExec = objShell.Exec("cmd.exe /C P:\SecurityLogs\Logparser.exe file:P:\SecurityLogs\EventIDDistrib_with_selected_event_ID.sql?sourcefile=P:\SecurityLogs\log\" & genmonth & genday & "t_Server1.txt+destfile=P:\SecurityLogs\BarChart\" & genmonth & genday & "Selected_Server1.csv -i:csv -o:csv") strPingResults = LCase(objExec.StdOut.ReadAll) Event Logs Review Process (3):  Event Logs Review Process (3) Add the header to the beginning of the file. Date,Time,EventID,SourceName,Dummy,Server,Description . They are used in the SQL query. SQL Query in the EventIDDistrib_with_selected event_ID.sql: SELECT StrCat(TO_STRING(EventID),Description) as EventID_And_Source,Count(*) as Total FROM %sourcefile% To %destfile% WHERE EventID in (529; 530; 531; 532; 535; 537; 539;608; 609; 612; 613; 614; 615; 616; 617; 620; 624; 625;626; 627; 628; 629; 630; 631; 632; 633; 634; 635; 636;637; 638; 639; 640; 641; 642; 643; 644; 645; 646; 647;648; 649; 650; 651; 652; 653; 654; 655; 656; 657; 658;659; 660; 661; 662; 663; 664; 665; 666; 667; 675; 676; 677) GROUP BY EventID_And_Source Final Log Review Report Element:  Final Log Review Report Element Scope and Content Which servers do we monitor? Grand Total Figures: It shows no. of logs for each event for every server Group the events into different categories: Appendix with detailed event statistics Statistics with no breakdown. Detailed statistics breakdown with log description Appendix of selected critical events Group the events for the management report:  Group the events for the management report Critical Events Selection:  Critical Events Selection Windows Server Event Different parties will have different risk ranking on particular event. Challenge again…:  Challenge again… How to justify the event is not suspicious? How to filter out those duplicated event logs further? We need to keep an eye on any new patch or/and updates, it may add some new event we have not captured. Current weakness:  Current weakness No co-relation could be drawn. No short-cut, all manual effort in investigation Incidence Response or Ignorance Response?:  Incidence Response or Ignorance Response? Again, log filtering and review is not a goal, we need to response any irregularities once it is discovered. One more mindset from CEO: “We are doing “XXXX” business, not “information security”! If the IR is reviewed once a year due to the coming of HKMA, generally, the staff and management awareness is lacking, it is hard to go ahead, may be, people may response with ignorance. Summary:  Summary Logs are here, but not the one you desire. Logs are obtained but useless if you don’t know how to review. Logs are reviewed but useless if there is no process to response. Shorten the Fault Detection Time. However, it needs management support. As an system architect, they may need to discuss with other architects and security officer about log facilities for better management and integration. Resources:  Resources An unofficial Log Parser support site http://www.logparser.com/ Log Parser Toolkit (book) http://www.syngress.com/catalog/?pid=3110 How to generate web-based report (I have discussed with others in http://www.logparser.com) , search “How to create a customized HTML report?” in the forum. Understanding Windows Logging http://www.windowsecurity.com/articles/Understanding_Windows_Logging.html Tutorial and Tools http://www.windowsecurity.com/pages/search.asp?query=Log+Monitoring Any questions?:  Any questions? Please feel free to share with me. You could reach me at [email protected] or [email protected]

Related presentations


Other presentations created by Taddeo

Famous GLBT People
15. 04. 2008
0 views

Famous GLBT People

app PCT 020906
11. 01. 2008
0 views

app PCT 020906

pptleadership 20041
12. 01. 2008
0 views

pptleadership 20041

resilience youth development
13. 01. 2008
0 views

resilience youth development

Indiana
09. 01. 2008
0 views

Indiana

Babies
15. 01. 2008
0 views

Babies

Faith Christian Church Picnic
15. 01. 2008
0 views

Faith Christian Church Picnic

Language Attitude
17. 01. 2008
0 views

Language Attitude

Confined Space Entry
17. 01. 2008
0 views

Confined Space Entry

KillerRobot2005
23. 01. 2008
0 views

KillerRobot2005

africancolonialism
10. 01. 2008
0 views

africancolonialism

vincent gulotta the wizards bowl
24. 01. 2008
0 views

vincent gulotta the wizards bowl

Soul of the Duelist
29. 01. 2008
0 views

Soul of the Duelist

cult branding
07. 02. 2008
0 views

cult branding

EGNRET26 Japan
13. 02. 2008
0 views

EGNRET26 Japan

The Epic Hero powerpoint
18. 02. 2008
0 views

The Epic Hero powerpoint

13 PHP intro
21. 02. 2008
0 views

13 PHP intro

10451
19. 01. 2008
0 views

10451

Kopecek ISC Tallinn
23. 01. 2008
0 views

Kopecek ISC Tallinn

biwebinar
05. 02. 2008
0 views

biwebinar

Chinese Politics
14. 02. 2008
0 views

Chinese Politics

9 Pandemic Influenza Reinhardt
20. 03. 2008
0 views

9 Pandemic Influenza Reinhardt

Early Human Culture
03. 04. 2008
0 views

Early Human Culture

UN Gobal Compact
13. 01. 2008
0 views

UN Gobal Compact

March 2007
04. 02. 2008
0 views

March 2007

espnclassic
14. 04. 2008
0 views

espnclassic

medication
17. 04. 2008
0 views

medication

chiral
22. 04. 2008
0 views

chiral

Lecture 3
22. 04. 2008
0 views

Lecture 3

energy e3 0711
24. 04. 2008
0 views

energy e3 0711

DarrylLuesby e
07. 05. 2008
0 views

DarrylLuesby e

Presentation Session 6
08. 05. 2008
0 views

Presentation Session 6

ancient
02. 05. 2008
0 views

ancient

modern art
03. 03. 2008
0 views

modern art

004
02. 05. 2008
0 views

004

AFPs
07. 02. 2008
0 views

AFPs

WorkstationErgo
07. 03. 2008
0 views

WorkstationErgo

WebStandards
30. 01. 2008
0 views

WebStandards

African Tale 22 September
24. 01. 2008
0 views

African Tale 22 September

elizabeth
06. 02. 2008
0 views

elizabeth

IPSec 4 ISAKMP IKE
08. 02. 2008
0 views

IPSec 4 ISAKMP IKE

13 chaturvedi
22. 01. 2008
0 views

13 chaturvedi

CoverttoOvert
04. 02. 2008
0 views

CoverttoOvert

p3430
04. 02. 2008
0 views

p3430

martyrdom final draft
16. 01. 2008
0 views

martyrdom final draft

august14
13. 01. 2008
0 views

august14

UrbanWoodUtilconf
12. 02. 2008
0 views

UrbanWoodUtilconf

Cotton Wool Kids LR
28. 01. 2008
0 views

Cotton Wool Kids LR

DDLCollectInternal
25. 01. 2008
0 views

DDLCollectInternal

IET RKF Y2 SanDiego CY 2002 CP
22. 01. 2008
0 views

IET RKF Y2 SanDiego CY 2002 CP

N6 Sutherst
16. 01. 2008
0 views

N6 Sutherst