I was celebrating my https://iislogs.com server has been up for 100 days by geeking on HTTP Modules / HTTP Handlers.  I’m still working through my HTTP module / handlers example.  Once I have the code finished, I’ll post up.  I found some good links for this topic.  Enjoy

ASP.NET Application Life Cycle Overview  (Nice graph of all events)
http://msdn2.microsoft.com/en-us/library/ms178473.aspx

Introduction to HTTP Handlers 
http://msdn2.microsoft.com/en-us/library/ms227675.aspx

Introduction to HTTP Modules 
http://msdn2.microsoft.com/en-us/library/ms178468.aspx

Custom ASP.NET Processing with HTTP Handlers 
http://msdn2.microsoft.com/en-us/library/5c67a8bd.aspx

ASP.NET 2.0 Internals 
http://msdn2.microsoft.com/en-us/library/ms379581(VS.80).aspx

appcmd list config /section:system.webServer/modules

appcmd list config /section:system.webServer/handlers

The first 100 days of IIS7

I’ve been bitten by the Powershell bug.  My scripting background tells me I should really learn this tool.  Since I’ve been writing console applications for a few years, mostly to do WMI and related Web service calls.  This should be fairly easy in Powershell.  As with anything, learning the syntax and how things are done is a bit frustrating at first.  There are tons of examples available, however if you are like me, you need to figure it out the hard way. Once you get the swing of it, things should be pretty straight forward.

After figuring out a couple of examples, which are posted below, I’m starting to get the swing of it.  There is a lot though I don’t understand but this will come in time. I miss intellisense that is built into Visual Studio.   The get-member cmdlet is useful in helping figure out which methods / properties are available to use.  Another command I like is man.  A few years back, I got into FreeBSD and used the ‘man’ command all the time.  Powershell has this available and it seems natural.  This is a basic blog, but helps me learn Powershell and share some a couple examples for future reference.  Happy Scripting!

Using MAN to pipe out information on get-member cmdlet

Type PS c:>man get-member

This is what is displayed,  a nice short version of help.

NAME
    Get-Member
   
SYNOPSIS
    Gets information about objects or collections of objects.
   
   
SYNTAX
    Get-Member [[-name] <string[]>] [-inputObject <psobject>] [-memberType {<AliasProperty> | <CodeProperty> | <Propert
    y> | <NoteProperty> | <ScriptProperty> | <Properties> | <PropertySet> | <Method> | <CodeMethod> | <ScriptMethod> |
    <Methods> | <ParameterizedProperty> | <MemberSet> | <All>}] [-static] [<CommonParameters>]
   
   
DETAILED DESCRIPTION
    Gets information about the members of objects. Get-Member can accept input from the pipeline or as the value of the
     InputObject parameter. You can use the MemberType parameter to specify the type of members you want information ab
    out.
   
    If you pipeline input to Get-Member, it outputs a MemberDefinition object for each distinct type of input object. 
    For example, if you pipe the output of Get-ChildItem to Get-Member in a directory that includes at least one subdir
    ectory and one file, it returns two MemberDefinition objects. One includes information about the FileInfo object an
    d the other includes information about the DirectoryInfo object. Get-Member outputs only two MemberDefinition objec
    ts, regardless of how many files or subdirectories are in the directory.
   
    The output of Get-Member is different if you supply input by using the InputObject parameter. In that case, Get-Mem
    ber returns a single MemberDefinition object that represents either the single input object or the collection class
     that contains the set of input objects.
   
    To retrieve information about static members, you must specify the Static parameter.
   

RELATED LINKS
    Add-Member
    Get-Help
    Get-Command
    Get-PSDrive

REMARKS
    For more information, type: "get-help Get-Member -detailed".
    For technical information, type: "get-help Get-Member -full".

Create a new object at the command line
$sw = new-object System.IO.StreamWriter("c:tempss.txt")

Type this to find out all methods available
$sw | get-member -memberType methods

PS C:temp> $sw | get-member -membertype methods

   TypeName: System.IO.StreamWriter

Name                      MemberType Definition
—-                      ———- ———-
Close                     Method     System.Void Close()
CreateObjRef              Method     System.Runtime.Remoting.ObjRef CreateObjRef(Type requestedType)
Dispose                   Method     System.Void Dispose()
Equals                    Method     System.Boolean Equals(Object obj)
Flush                     Method     System.Void Flush()
GetHashCode               Method     System.Int32 GetHashCode()
GetLifetimeService        Method     System.Object GetLifetimeService()
GetType                   Method     System.Type GetType()
get_AutoFlush             Method     System.Boolean get_AutoFlush()
get_BaseStream            Method     System.IO.Stream get_BaseStream()
get_Encoding              Method     System.Text.Encoding get_Encoding()
get_FormatProvider        Method     System.IFormatProvider get_FormatProvider()
get_NewLine               Method     System.String get_NewLine()
InitializeLifetimeService Method     System.Object InitializeLifetimeService()
set_AutoFlush             Method     System.Void set_AutoFlush(Boolean value)
set_NewLine               Method     System.Void set_NewLine(String value)
ToString                  Method     System.String ToString()
Write                     Method     System.Void Write(Char value), System.Void Write(Char[] buffer), System.Void Wr..
WriteLine                 Method     System.Void WriteLine(), System.Void WriteLine(Char value), System.Void WriteLi..

‘Create a web request
$request = [System.Net.WebRequest]::Create("http://www.iislogs.com/testlink.aspx")
$response = $request.GetResponse()
$requestStream = $response.GetResponseStream()
$readStream = new-object System.IO.StreamReader $requestStream
new-variable db
$db = $readStream.ReadToEnd()
$readStream.Close()
$response.Close()

‘Create a new file and write the contents of that to a file
$sw = new-object system.IO.StreamWriter("c:tempss2.txt")
$sw.writeline($db)
$sw.close()
get-content "c:tempss.txt"

Take care,

Steve Schofield
Microsoft MVP – IIS

I’m on a mission!  I have to search log files that are between 150 MB and larger.  These are syslogd files generated.  Here is sample output.

2023-01-15 00:00:10 Mail.Debug 111.111.111.111 join[00000]: server.steve.net[1.1.1.1] 1168837209-0ff301650000-3Bdfku 1168837209 1168837210 SCAN – [email protected] [email protected] – 2 39 *abcdef.xyz.com SUBJ:Blah this is replaced

I don’t confess to be an expert but this has got to be easier than I’m making it.  I want to share my experiences, so far and I love adventures like this, because I learn a lot. What am I after?  I want the ‘server.steve.net[1.1.1.1]’ or just [1.1.1.1].  

Example #1 – For small files this works good

$sb =  new-object System.Text.StringBuilder
$re = new-object regex(‘[(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})]’)
$m = $re.match((get-content mySysLogfile.txt))
while ($m.Success)
{
$sb.Append($m.value)
$sb.AppendLine()
$m = $m.NextMatch()
}
$sb.ToString() > st1.txt

Example #2 – Works for large files extracting the data, performance takes a couple hours.

$sb =  new-object System.Text.StringBuilder
$re = new-object regex(‘[(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})]’)
$m = [System.IO.File]::OpenText("d:tempsyslogcatchall16.txt")
while($line = $m.ReadLine())
{
$line = $re.Match($line)
$sb.Append($line)
$sb.AppendLine()
}
$m.Close()
$sb.ToString() > st1.txt

Log Parser example

‘Example 1
logparser -i:tsv "select top 50 Count(extract_token(field6,1, ‘[‘)) as CountOfIt,extract_token(Field6,1,'[‘) as IPAddress into Steve.csv from  ‘\ServerNameShareNamesyslogd11.txt’ Group By IPA
ddress order by CountOfIt DESC" -headerRow:off -iSeparator:’spaces’

‘Example 2
logparser -i:tsv "select Top 10 Count(field6),Field6 from\ServerNameShareNamesyslogd11.txt Group By Field6" -headerRow:off -iSeparator:’spaces’

‘Findstr
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/findstr.mspx?mfr=true

‘QGrep
http://www.ss64.com/nt/qgrep.html

In conclusion, the clear winner was Log parser, speed and accuracy were great.  Powershell was ‘cool’ but took too long.  Maybe as I get better at Powershell, that will change.  Findstr & QGrep appear to be more for parsing out entire lines of text.  That was my experience, it could be my lack of advanced knowledge with these tools.  I use FINDSTR a lot for doing quick searches, it is faster than FIND.  I was hoping to use regular expressions, but found Powershell was easier to use for regex.  I didn’t try a grep utility found on sourceforge, because Log Parser did the trick.  If you have other experiences using FINDSTR, QGrep or some other tool, please pass them along.  Hope this helps!

The Debug Diagnostic Tool (DebugDiag) is designed to assist in troubleshooting issues such as hangs, slow performance, memory leaks or fragmentation, and crashes in any Win32 user-mode process.

http://www.microsoft.com/downloads/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&DisplayLang=en

This is very handy if you are responsible for troubleshooting IIS and related application issues.

I was troubleshooting an issue where I was sending an email using .NET 1.1 and attaching PDF files. The PDF files where getting an error when trying to open the attachment.  Here is what was displayed in Adobe reader.  "There was an error processing a page. A file I/O error has occurred".  Thanks a bunch to Dave Wanta, creator of http://ASPNetEmail.com for providing a great resource about System.Web.Mail ( http://www.systemwebmail.com ).  A similar example worked in ASP.NET 2.0, comparing email headers provided the clues it was how the attachment was getting encoded.  I found the answer in the article listed below.

This article explains it.
http://systemwebmail.com/faq/4.4.8.aspx 

The bolded code is what you have to alter

<%@ Page Language="C#" %>
<%@ Import Namespace="System.Web" %>
<%@ Import Namespace="System.Web.Mail" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<script runat="server">

protected void Page_Load(object sender, EventArgs e)
    {

    string FilePath = Request.PhysicalApplicationPath + "/c.pdf";
    MailMessage msgMail = new MailMessage();

    msgMail.To = "[email protected]";
    msgMail.From = "[email protected]";
    msgMail.Subject = "Your Favorite Subject";
    msgMail.Body = @"Some Text goes here";
    msgMail.Attachments.Add(new MailAttachment(FilePath, MailEncoding.UUEncode));
    SmtpMail.Send(msgMail);
    Response.Write("Thank you."); 
}

</script>

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
    <title>Untitled Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
   
    </div>
    </form>
</body>
</html>

Hope this helps

Steve Schofield
Microsoft MVP – IIS

Here is a command using Log parser to retrieve a specific value in the cs-uri-query item.

LogParser -I:iisw3c "SELECT * INTO Filename.csv FROM \ServerName\ServerNameShareNameW3SVC111*.log where cs-uri-query Like ‘%http://bc0.cn/1.js%’" -o:CSV

If longhorn RTM runs as well as RC1 has, things are going to be really awesome!  My RC1 (version 5600) Longhorn server has been up for 86 days and counting.  I’ll put some stats together to see how many requests https://iislogs.com has served.  Memory usage has stayed at about 80% to 85% memory usage.  Not bad for a desktop pc with 512 MB shared memory.   I wanted to pass this along to let people know how stable IIS7 and Longhorn is.  Exciting things to come! 

‘Here is a picture showing the uptime.

I found an article that explains in further detail how to make the "404 Not Found" show-up even when using a custom error page.  You need to add <% response.status = "404 Not Found" %> to your custom 404 webpage and the appropriate status code is returned. 

http://web-sniffer.net/ tool.

‘Here is a link to the article
http://www.mcanerin.com/EN/articles/301-redirect-404-error.asp

‘Status codes KB article.
http://support.microsoft.com/kb/318380

Found this blog by Tobin Titus.  Definitely check it out!

http://blogs.iis.net/tobintitus/archive/2006/11/30/powershell-and-microsoft-web-administration.aspx

http://blogs.iis.net/tobintitus/archive/2006/12/01/more-on-iis-7-administration-with-powershell.aspx

Tx,

Steve Schofield

I’m following a thread in the newsgroups, actually I replied a few times about setting Logging options at a global level.  Unlike previous versions of client OS’s, Vista allows more than 1 site.  One thing that I couldn’t find was the Logging options in Vista IIS manager.  I can set this with appcmd and manually set in applicationHost.config.   There are 2 sections I could find, <system.applicationHostlog> section and under <sites/site/siteDefault>  Here are some raw notes I had.  I tried to clean up as best as possible.  One of my favorite appcmd command is "appcmd set config -section:XXXX -?   (XXXX represents what section)"  This pipes out a list of properties for that section.  It was a good evening digging into logging and Vista!   Hope you find this useful.

Example 1
C:WindowsSystem32inetsrv>appcmd list config -section:log
<system.applicationHost>
  <log>
    <centralBinaryLogFile enabled="true" directory="%SystemDrive%inetpublogsLogFiles" />
    <centralW3CLogFile enabled="true" directory="%SystemDrive%inetpublogsLogFiles" />
  </log>
</system.applicationHost>

Example #2
C:WindowsSystem32inetsrv>appcmd set config -section:log -?
ERROR ( message:-logInUTF8
-centralLogFileMode
-centralBinaryLogFile.enabled
-centralBinaryLogFile.directory
-centralBinaryLogFile.period
-centralBinaryLogFile.truncateSize
-centralBinaryLogFile.localTimeRollover
-centralW3CLogFile.enabled
-centralW3CLogFile.directory
-centralW3CLogFile.period
-centralW3CLogFile.truncateSize
-centralW3CLogFile.localTimeRollover
-centralW3CLogFile.logExtFileFlags
 )

‘Set the log default path
C:WindowsSystem32inetsrv>appcmd set config -section:log /centralW3CLogFile.directory:c:wwwlogs
Applied configuration changes to section "system.applicationHost/log" for "MACHINE/WEBROOT/APPHOST" at configuration commit path "MACHINE/WEBROOT/APPHOST"

‘Set the log default output
C:WindowsSystem32inetsrv>appcmd set config -section:log /logInUTF8:true
Applied configuration changes to section "system.applicationHost/log" for "MACHNE/WEBROOT/APPHOST" at configuration commit path "MACHINE/WEBROOT/APPHOST"

C:WindowsSystem32inetsrv>appcmd list config -section:log
<system.applicationHost>
  <log logInUTF8="true">
    <centralBinaryLogFile enabled="true" directory="%SystemDrive%inetpublogsLogFiles" />
    <centralW3CLogFile enabled="true" directory="c:wwwlogs" />
  </log>
</system.applicationHost>

Example #3
C:WindowsSystem32inetsrv>appcmd set config -section:sites -?
ERROR ( message:-siteDefaults.name
-siteDefaults.id
-siteDefaults.serverAutoStart
-siteDefaults.bindings.[protocol=’string’,bindingInformation=’string’].protocol
-siteDefaults.bindings.[protocol=’string’,bindingInformation=’string’].bindingInformation
-siteDefaults.limits.maxBandwidth
-siteDefaults.limits.maxConnections
-siteDefaults.limits.connectionTimeout
-siteDefaults.logFile.logExtFileFlags
-siteDefaults.logFile.customLogPluginClsid
-siteDefaults.logFile.logFormat
-siteDefaults.logFile.directory
-siteDefaults.logFile.period
-siteDefaults.logFile.truncateSize
-siteDefaults.logFile.localTimeRollover
-siteDefaults.logFile.enabled
-siteDefaults.traceFailedRequestsLogging.enabled
-siteDefaults.traceFailedRequestsLogging.directory
-siteDefaults.traceFailedRequestsLogging.maxLogFiles
-applicationDefaults.path
-applicationDefaults.applicationPool
-applicationDefaults.enabledProtocols
-virtualDirectoryDefaults.path
-virtualDirectoryDefaults.physicalPath
-virtualDirectoryDefaults.userName
-virtualDirectoryDefaults.password
-virtualDirectoryDefaults.logonMethod
-virtualDirectoryDefaults.allowSubDirConfig
-[name=’string’,id=’unknown’].name
-[name=’string’,id=’unknown’].id
-[name=’string’,id=’unknown’].serverAutoStart
-[name=’string’,id=’unknown’].bindings.[protocol=’string’,bindingInformation=’string’].protocol
-[name=’string’,id=’unknown’].bindings.[protocol=’string’,bindingInformation=’string’].bindingInformation
-[name=’string’,id=’unknown’].limits.maxBandwidth
-[name=’string’,id=’unknown’].limits.maxConnections
-[name=’string’,id=’unknown’].limits.connectionTimeout
-[name=’string’,id=’unknown’].logFile.logExtFileFlags
-[name=’string’,id=’unknown’].logFile.customLogPluginClsid
-[name=’string’,id=’unknown’].logFile.logFormat
-[name=’string’,id=’unknown’].logFile.directory
-[name=’string’,id=’unknown’].logFile.period
-[name=’string’,id=’unknown’].logFile.truncateSize
-[name=’string’,id=’unknown’].logFile.localTimeRollover
-[name=’string’,id=’unknown’].logFile.enabled
-[name=’string’,id=’unknown’].traceFailedRequestsLogging.enabled
-[name=’string’,id=’unknown’].traceFailedRequestsLogging.directory
-[name=’string’,id=’unknown’].traceFailedRequestsLogging.maxLogFiles
-[name=’string’,id=’unknown’].applicationDefaults.path
-[name=’string’,id=’unknown’].applicationDefaults.applicationPool
-[name=’string’,id=’unknown’].applicationDefaults.enabledProtocols
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.path
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.physicalPath
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.userName
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.password
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.logonMethod
-[name=’string’,id=’unknown’].virtualDirectoryDefaults.allowSubDirConfig
-[name=’string’,id=’unknown’].[path=’string’].path
-[name=’string’,id=’unknown’].[path=’string’].applicationPool
-[name=’string’,id=’unknown’].[path=’string’].enabledProtocols
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.path
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.physicalPath
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.userName
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.password
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.logonMethod
-[name=’string’,id=’unknown’].[path=’string’].virtualDirectoryDefaults.allowSubDirConfig
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].path
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].physicalPath
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].userName
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].password
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].logonMethod
-[name=’string’,id=’unknown’].[path=’string’].[path=’string’].allowSubDirConfig
 )

‘Set the default log path on the siteDefault section
C:WindowsSystem32inetsrv>appcmd set config -section:sites /siteDefaults.logFile.directory:c:wwwlogs
Applied configuration changes to section "system.applicationHost/sites" for "MACHINE/WEBROOT/APPHOST" at configuration commit path "MACHINE/WEBROOT/APPHOST"

Log schema in the IIS_schema.xml

 <sectionSchema name="system.applicationHost/log">
    <attribute name="logInUTF8" type="bool" defaultValue="true" />
    <attribute name="centralLogFileMode" type="enum" defaultValue="Site" >
      <enum name="Site" value="0"/>
      <enum name="CentralBinary" value="1"/>
      <enum name="CentralW3C" value="2"/>
    </attribute>
    <element name="centralBinaryLogFile">
      <attribute name="enabled" type="bool" defaultValue="false" />
      <attribute name="directory" type="string" defaultValue="%SystemDrive%inetpublogsLogFiles" />
      <attribute name="period" type="enum" defaultValue="Daily">
        <enum name="Hourly" value="4"/>
        <enum name="Daily" value="1"/>
        <enum name="Weekly" value="2"/>
        <enum name="Monthly" value="3"/>
        <enum name="MaxSize" value="0"/>
      </attribute>
      <attribute name="truncateSize" type="int64" defaultValue="20971520" validationType="integerRange" validationParameter="1048576,4294967295" />
      <attribute name="localTimeRollover" type="bool" defaultValue="false"/>
    </element>
    <element name="centralW3CLogFile">
      <attribute name="enabled" type="bool" defaultValue="true" />
      <attribute name="directory" type="string" expanded="true" defaultValue="%SystemDrive%inetpublogsLogFiles" validationType="nonEmptyString" />
      <attribute name="period" type="enum" defaultValue="Daily">
        <enum name="Hourly" value="4"/>
        <enum name="Daily" value="1"/>
        <enum name="Weekly" value="2"/>
        <enum name="Monthly" value="3"/>
        <enum name="MaxSize" value="0"/>
      </attribute>
      <attribute name="truncateSize" type="int64" defaultValue="20971520" validationType="integerRange" validationParameter="1048576,4294967295" />
      <attribute name="localTimeRollover" type="bool" defaultValue="false"/>
      <attribute name="logExtFileFlags" type="flags" defaultValue="Date, Time, ClientIP, UserName, SiteName, ServerIP, Method, UriStem, UriQuery, HttpStatus, Win32Status, ServerPort, UserAgent, HttpSubStatus">
        <flag name="Date" value="1"/>
        <flag name="Time" value="2"/>
        <flag name="ClientIP" value="4"/>
        <flag name="UserName" value="8"/>
        <flag name="SiteName" value="16"/>
        <flag name="ComputerName" value="32"/>
        <flag name="ServerIP" value="64"/>
        <flag name="Method" value="128"/>
        <flag name="UriStem" value="256"/>
        <flag name="UriQuery" value="512"/>
        <flag name="HttpStatus" value="1024"/>
        <flag name="Win32Status" value="2048"/>
        <flag name="BytesSent" value="4096"/>
        <flag name="BytesRecv" value="8192"/>
        <flag name="TimeTaken" value="16384"/>
        <flag name="ServerPort" value="32768"/>
        <flag name="UserAgent" value="65536"/>
        <flag name="Cookie" value="131072"/>
        <flag name="Referer" value="262144"/>
        <flag name="ProtocolVersion" value="524288"/>
        <flag name="Host" value="1048576"/>
        <flag name="HttpSubStatus" value="2097152"/>
      </attribute>
    </element>
  </sectionSchema>