Controling SOAP security header “mustUnderstand” attribute in WCF client

I was recent tasked with connecting .NET WCF with an older java soap service.  Unfortunately the service was not standard and needed more WCF customization than normal, starting with the security.

I needed to include both a client x509 certificate for the Transport tunnel, and a username/password combination in the soap header itself.  By playing around with the web.config binding configurations I could have eventually achieved this.  However that was not the end of the customizations it needed.

This service needed the EncodedMustUnderstand flag set to false for the security header.  Back with WSE that used to be a very simple flag to set, but for some reason Microsoft decided to remove that capability from WCF.  One guy claimed to find a simple fix for this, however his fix did not work for me.

Throughout the years since WCF has been introduced many people have had this or very similar issues when dealing with Java based SOAP services.  I have yet to find someone who has discovered a good solution.  In my own two week journey to solving this issue I had to do some pretty serious customization to WCF.  By sharing my discoveries here I hope to help other people shorted the time they spend enduring the same pain.

The first thing I did was to create an extension method for the service allowing me to create a CustomBinding for it.  This method served as the hub to which I attached various other classes and methods as I built them.  It seems daunting, and it really is.  Separating out the various ideas into their own files was the approach I took to keeping the major areas of functionality separate in my head.

Base Helper Class

using System;
using System.Configuration;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Security;
using System.ServiceModel.Security.Tokens;

namespace Services
{
    public static class ClientHelper
    {

        public static PClient Initialize(this PClient client)
        {
            var urlConfig = ConfigurationManager.AppSettings["Url"].ToString();

            if (string.IsNullOrEmpty(urlConfig))
                throw new Exception("Missing Url Config");

            var url = new UrlSecurityConfig(urlConfig);

            //System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;

            var security = SecurityBindingElement.CreateCertificateOverTransportBindingElement();
            security.IncludeTimestamp = true;
            security.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic256;
            security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10;
            security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
            security.SecurityHeaderLayout = SecurityHeaderLayout.Lax;
            security.EnableUnsecuredResponse = true;

            //security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11;
            //security.EndpointSupportingTokenParameters.SignedEncrypted.Add(new X509SecurityTokenParameters(X509KeyIdentifierClauseType.Any, SecurityTokenInclusionMode.AlwaysToRecipient)); // add specific x509 cert security            security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
            //security.DefaultAlgorithmSuite = new Basic128Sha256Rsa15Sha1AlgorithmSuite(); // when we need to tweak the security suite


            //var encoding = new TextMessageEncodingBindingElement();
            //encoding.MessageVersion = MessageVersion.CreateVersion(EnvelopeVersion.Soap11, AddressingVersion.None);// MessageVersion.Soap11;
            ////encoding.MessageVersion = MessageVersion.Soap11WSAddressingAugust2004;
            //encoding.WriteEncoding = Encoding.UTF8;


            var transport = new HttpsTransportBindingElement();
            transport.MaxReceivedMessageSize = 20000000; // 20 megs
            transport.RequireClientCertificate = false;
            transport.TransferMode = TransferMode.Buffered;
            transport.DecompressionEnabled = false;

            CustomBinding binding = new CustomBinding();
            binding.Elements.Add(security);
            //binding.Elements.Add(encoding); // add normal encoding
            binding.Elements.Add(new MustUnderstandOffBindingElement("UTF-8", "text/xml", MessageVersion.Soap11)); // add custom encoder
            binding.Elements.Add(transport);

            var x509Config = ConfigurationManager.AppSettings["x509"].ToString();
            var X509Cert = Common.Get509Cert(x509Config);

            // THIS IS WHEN THE CERT NEEDS A SPECIFIC DOMAIN IDENTITY SPECIFIED
            //var identity = EndpointIdentity.CreateX509CertificateIdentity(X509Cert);
            //var address = new EndpointAddress(new Uri(url), identity);
            ////var binding = new WSHttpBinding(SecurityMode.Message);
            ////binding.Security.Message.ClientCredentialType = MessageCredentialType.Certificate;
            //var factory = new ChannelFactory<CoAServices_v1r21_P>(binding, address);
            ////factory.Endpoint.EndpointBehaviors.Remove(typeof(System.ServiceModel.Description.ClientCredentials));
            ////factory.Endpoint.EndpointBehaviors.Add(new Services.WsNonceCustomCredentials());
            //factory.Credentials.UserName.UserName = username;
            //factory.Credentials.UserName.Password = password;
            //factory.Credentials.ClientCertificate.Certificate = X509Cert;
            //factory.Credentials.ServiceCertificate.DefaultCertificate = X509Cert;

            client.Endpoint.Binding = binding;
            client.Endpoint.Address = new EndpointAddress(url.Url);
            //client.Endpoint.EndpointBehaviors.Add(new ExClientBehavior()); // adding a custom behavior built in a custom class
            //client.Endpoint.EndpointBehaviors.Add(new MustUnderstandBehavior(false)); // create a behavior with the MustUnderstand attribute set to false, does not work for headers

            //// replace normal credencials with custom ones that generate Nonce:
            //client.ChannelFactory.Endpoint.Behaviors.Remove<System.ServiceModel.Description.ClientCredentials>();
            //client.ChannelFactory.Endpoint.Behaviors.Add(new Services.WsNonceCustomCredentials());

            client.ClientCredentials.UserName.UserName = url.UserName;
            client.ClientCredentials.UserName.Password = url.Password;

            client.ClientCredentials.ClientCertificate.Certificate = X509Cert;
            //client.ClientCredentials.ServiceCertificate.DefaultCertificate = X509Cert;
            //client.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.None;
            //client.ClientCredentials.ServiceCertificate.Authentication.RevocationMode = X509RevocationMode.NoCheck;

            return client;
            //return factory.CreateChannel();
        }

    }


}

As you can see, there is lots of commented out code to plug in various pieces of functionality.  I ended up not needing half of it, but there was no point in throwing away code that functioned.  For the purposes of this post, only the commented out ExClientBehavior lines and the custom Nonce credencials will be discussed in addition to the working lines of code.

One of the first steps I took was to create a helper class with a method to get my x509 certificate.  I later added a method to do some basic url config parsing.  This ended up being the easiest part of the whole process.

EDIT: A quick side note, when generating the code below for this blog post, it kept wanting to rename X509Certificate2Collection and other class names as 17bH1SYLoBdGsBaDedPR2EE3JUt8oRS7qd.  A very odd quirk, but I have seen that weird name in several Microsoft posts as well, so apparently I am not the only one who has had that issue when writing posts. 

EDIT2: It turns out the weird renaming was caused by a Chrome extension I am running called uBlock.  So if you see something similar, then check your extensions.  In my case, white listing the blogger domain fixed the issue.

Common Helper Methods

using System;
using System.Linq;
using System.Security.Cryptography.X509Certificates;

namespace Services
{
    public static class Common
    {
        public static X509Certificate2 Get509Cert(string LocationConfig)
        {
            var locationParts = LocationConfig.Split(':');
            if (locationParts.Length < 2)
                throw new Exception("Invalid X509 Config");

            X509Certificate2 cer = new X509Certificate2();
            X509Store store = locationParts[0] == "CurrentUser" ? new X509Store(StoreLocation.CurrentUser) : new X509Store(StoreLocation.LocalMachine);

            store.Open(OpenFlags.ReadOnly);

            X509Certificate2Collection cers = store.Certificates.Find(X509FindType.FindBySubjectName, locationParts[1], false);
            if (cers.Count == 0)
            {
                throw new Exception("Can't find X509 Cert.");
            }

            var cert = cers[0];

            store.Close();
            return cert;
        }
    }

    public class UrlSecurityConfig
    {
        public string Url { get; protected set; }
        public string UserName { get; protected set; }
        public string Password { get; protected set; }

        public UrlSecurityConfig(string config)
        {
            if(!config.Contains('@'))
            {
                Url = config;
                return;
            }

            var uendStart = config.LastIndexOf('@');
            var uStart = config.Substring(0, config.IndexOf('/') + 2);
            var uEnd = config.Substring(uendStart + 1);
            Url = uStart + uEnd;

            var nameEnd = config.IndexOf(':', uStart.Length + 1);
            UserName = config.Substring(uStart.Length, nameEnd - uStart.Length);
            Password = config.Substring(nameEnd + 1, uendStart - nameEnd - 1);
        }
    }
}

After trying all sorts of ways to get into the security header using existing settings, I finally decided to write my own custom Behavior.  This code I copied directly from someone else's post who was struggling with something very similar.  Unfortunately, I got a rude shock when I found out that the security headers are not part of the request sent to the behavior.

Without access to those I was unable to modify them, although in testing I discovered that I was unable to affect the output of the headers here anyway despite the code looking like it was working correctly when I stepped through it.  Perhaps I still have something slightly off.

Custom Behavior Classes

using System;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.ServiceModel.Configuration;

namespace Services
{
    public class ExClientBehavior : IEndpointBehavior
    {
        #region IEndpointBehavior Members

        public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
        {
            // no op
        }

        public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
        {
            ExInspector inspector = new ExInspector();
            clientRuntime.MessageInspectors.Add(inspector);
            //clientRuntime.ValidateMustUnderstand = false;

        }

        public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
        {

            // no op
        }

        public void Validate(ServiceEndpoint endpoint)
        {
            // no op
        }

        #endregion
    }


    public class ExClientBehaviorExtensionElement : BehaviorExtensionElement
    {
        public override Type BehaviorType
        {
            get { return typeof(ExClientBehavior); }
        }

        protected override object CreateBehavior()
        {
            return new ExClientBehavior();
        }
    }

    public class ExInspector : IClientMessageInspector
    {

        #region IClientMessageInspector Members

        public void AfterReceiveReply(ref Message reply, object correlationState)
        {
            // no op
            return;
        }

        public object BeforeSendRequest(ref Message request, IClientChannel channel)
        {
            MessageBuffer buffer = request.CreateBufferedCopy(int.MaxValue);

            Message newMessage = buffer.CreateMessage();

            while (newMessage.Headers.Count > 0)
                newMessage.Headers.RemoveAt(0);
            
            foreach (var info in request.Headers)
            {
                newMessage.Headers.Add(MessageHeader.CreateHeader
                    (
                        info.Name,
                        info.Namespace,
                        string.Empty,
                        false,
                        string.Empty,
                        info.Relay
                    )
                );
            }


            request = newMessage;

            return null;
        }

        #endregion
    }
}

My next approach was to try writing my own Custom Encoder.  Many people had stated that was the way to get at the stream both as it was heading out the door, and as it was coming back in before it hit the service code.  I copied Microsoft's code and directions from here, here, and here.

I immediately started running into problems due to that class name rewrite in Microsoft's web posts that I mentioned earlier.  It took me a little while to create names that all worked and referenced each other correctly.

I could have handled the pain it had taken to get this far, but it wasn't over yet.  Microsoft's code has a flaw in it that resulted in a loop which consistently generated stack overflow errors.  I later discovered that the error centered around the "this.factory = factory;" line.  It generated an internal reference to the parent object that the parent then referenced causing a recursive infinite loop.

If there had been any sort of decent error that would have been easy to find and work through.  Unfortunately, these were the errors and Inner Exceptions that it kept generating:

Message: An error occurred while making the HTTP request to https://service. This could be due to the fact that the server certificate is not configured properly with HTTP.SYS in the HTTPS case. This could also be caused by a mismatch of the security binding between the client and the server.

Message: The underlying connection was closed: An unexpected error occurred on a send.

Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

Stepping through the code did show me that a stack overflow was occurring, but it was not obvious that it was a recursive reference that was causing it.  And, since my code had been copied from an official Microsoft website, my first instinct was to trust it and look elsewhere.  So I spent a long time looking at the exceptions and trying to figure out what they could mean.

During this time I spent a lot of time in Fiddler working with the returning soap message and tweaking it to see how it would affect .net.  I discovered that it is actually possible to strip off the characters surrounding the soap envelope that trigger the HTTP.SYS error in .net.  This allowed me to test the service even when sending from an HTTP source rather than the required HTTPS source.

I finally ended up with this working code.  You will notice that the primary difference from Microsoft's classes is the replacement for the "mustUnderstand" line, which was the hack I put in to toggle that flag as the message went out the door.

Custom Encoder Classes

using System;
using System.Text;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.IO;
using System.Xml;

namespace Services
{
    public class MustUnderstandOffBindingElement : MustUnderstandOffBindingElement, IWsdlExportExtension
    {
        private MessageVersion msgVersion;
        private string mediaType;
        private string encoding;
        private XmlDictionaryReaderQuotas readerQuotas;

        MustUnderstandOffBindingElement(MustUnderstandOffBindingElement binding)
            : this(binding.Encoding, binding.MediaType, binding.MessageVersion)
        {
            this.readerQuotas = new XmlDictionaryReaderQuotas();
            binding.ReaderQuotas.CopyTo(this.readerQuotas);
        }

        public MustUnderstandOffBindingElement(string encoding, string mediaType,
            MessageVersion msgVersion)
        {
            if (encoding == null)
                throw new ArgumentNullException("encoding");

            if (mediaType == null)
                throw new ArgumentNullException("mediaType");

            if (msgVersion == null)
                throw new ArgumentNullException("msgVersion");

            this.msgVersion = msgVersion;
            this.mediaType = mediaType;
            this.encoding = encoding;
            this.readerQuotas = new XmlDictionaryReaderQuotas();
        }

        public MustUnderstandOffBindingElement(string encoding, string mediaType)
            : this(encoding, mediaType, MessageVersion.Soap11WSAddressing10)
        {
        }

        public MustUnderstandOffBindingElement(string encoding)
            : this(encoding, "text/xml")
        {

        }

        public MustUnderstandOffBindingElement()
            : this("UTF-8")
        {
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return this.msgVersion;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.msgVersion = value;
            }
        }


        public string MediaType
        {
            get
            {
                return this.mediaType;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.mediaType = value;
            }
        }

        public string Encoding
        {
            get
            {
                return this.encoding;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.encoding = value;
            }
        }

        // This encoder does not enforces any quotas for the unsecure messages. The 
        // quotas are enforced for the secure portions of messages when this encoder
        // is used in a binding that is configured with security. 
        public XmlDictionaryReaderQuotas ReaderQuotas
        {
            get
            {
                return this.readerQuotas;
            }
        }

        #region IMessageEncodingBindingElement Members
        public override MessageEncoderFactory CreateMessageEncoderFactory()
        {
            return new MustUnderstandOffEncoderFactory(this.MediaType,
                this.Encoding, this.MessageVersion);
        }

        #endregion


        public override BindingElement Clone()
        {
            return new MustUnderstandOffBindingElement(this);
        }

        public override IChannelFactory<TChannel> BuildChannelFactory<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.BuildInnerChannelFactory<TChannel>();
        }

        public override bool CanBuildChannelFactory<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            return context.CanBuildInnerChannelFactory<TChannel>();
        }

        public override IChannelListener<TChannel> BuildChannelListener<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.BuildInnerChannelListener<TChannel>();
        }

        public override bool CanBuildChannelListener<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.CanBuildInnerChannelListener<TChannel>();
        }

        public override T GetProperty<T>(BindingContext context)
        {
            if (typeof(T) == typeof(XmlDictionaryReaderQuotas))
            {
                return (T)(object)this.readerQuotas;
            }
            else
            {
                return base.GetProperty<T>(context);
            }
        }

        #region IWsdlExportExtension Members

        void IWsdlExportExtension.ExportContract(WsdlExporter exporter, WsdlContractConversionContext context)
        {
        }

        void IWsdlExportExtension.ExportEndpoint(WsdlExporter exporter, WsdlContractConversionContext context)
        {
            // The MessageEncodingBindingElement is responsible for ensuring that the WSDL has the correct
            // SOAP version. We can delegate to the WCF implementation of TextMessageEncodingBindingElement for this.
            TextMessageEncodingBindingElement mebe = new TextMessageEncodingBindingElement();
            mebe.MessageVersion = this.msgVersion;
            ((IWsdlExportExtension)mebe).ExportEndpoint(exporter, context);
        }

        #endregion
    }

    public class MustUnderstandOffEncoderFactory : MessageEncoderFactory
    {
        private MessageEncoder encoder;
        private MessageVersion version;
        private string mediaType;
        private string charSet;

        internal MustUnderstandOffEncoderFactory(string mediaType, string charSet, MessageVersion version)
        {
            this.version = version;
            this.mediaType = mediaType;
            this.charSet = charSet;
            this.encoder = new MustUnderstandOffEncoder(this);

        }

        public override MessageEncoder Encoder
        {
            get
            {
                return this.encoder;
            }
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return this.version;
            }
        }

        internal string MediaType
        {
            get
            {
                return this.mediaType;
            }
        }

        internal string CharSet
        {
            get
            {
                return this.charSet;
            }
        }
    }

    public class MustUnderstandOffEncoder : MessageEncoder
    {
        private XmlWriterSettings writerSettings;
        private string contentType;

        private string _MediaType;
        private MessageVersion _MessageVersion;

        public MustUnderstandOffEncoder(MustUnderstandOffEncoderFactory factory)
        {
            _MediaType = factory.MediaType;
            _MessageVersion = factory.MessageVersion;

            this.writerSettings = new XmlWriterSettings();
            this.writerSettings.Encoding = Encoding.GetEncoding(factory.CharSet);
            //this.writerSettings.ConformanceLevel = ConformanceLevel.Fragment;
            //this.writerSettings.OmitXmlDeclaration = true;
            this.contentType = string.Format("{0}; charset={1}",
                        _MediaType, this.writerSettings.Encoding.HeaderName);
        }

        public override string ContentType
        {
            get
            {
                return this.contentType;
            }
        }

        public override string MediaType
        {
            get
            {
                return _MediaType;
            }
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return _MessageVersion;
            }
        }

        public override bool IsContentTypeSupported(string contentType)
        {
            if (base.IsContentTypeSupported(contentType))
            {
                return true;
            }
            if (contentType.Length == this.MediaType.Length)
            {
                return contentType.Equals(this.MediaType, StringComparison.OrdinalIgnoreCase);
            }
            else
            {
                if (contentType.StartsWith(this.MediaType, StringComparison.OrdinalIgnoreCase)
                    && (contentType[this.MediaType.Length] == ';'))
                {
                    return true;
                }
            }
            return false;
        }

        public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
        {
            byte[] msgContents = new byte[buffer.Count];
            Array.Copy(buffer.Array, buffer.Offset, msgContents, 0, msgContents.Length);
            bufferManager.ReturnBuffer(buffer.Array);

            MemoryStream stream = new MemoryStream(msgContents);
            return ReadMessage(stream, int.MaxValue);
        }

        public override Message ReadMessage(Stream stream, int maxSizeOfHeaders, string contentType)
        {
            XmlReader reader = XmlReader.Create(stream);
            return Message.CreateMessage(reader, maxSizeOfHeaders, this.MessageVersion);
        }

        public override ArraySegment<byte> WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
        {
            MemoryStream stream = new MemoryStream();
            XmlWriter writer = XmlWriter.Create(stream, this.writerSettings);
            message.WriteMessage(writer);
            writer.Close();

            string decoded = Encoding.UTF8.GetString(stream.ToArray());
            decoded = decoded.Replace("mustUnderstand=\"1\"", "mustUnderstand=\"0\"");
            stream.Close();
            byte[] messageBytes = System.Text.Encoding.UTF8.GetBytes(decoded);
            int messageLength = messageBytes.Length;

            //byte[] messageBytes = stream.GetBuffer();
            //int messageLength = (int)stream.Position;
            //stream.Close();

            int totalLength = messageLength + messageOffset;
            byte[] totalBytes = bufferManager.TakeBuffer(totalLength);
            Array.Copy(messageBytes, 0, totalBytes, messageOffset, messageLength);

            ArraySegment<byte> byteArray = new ArraySegment<byte>(totalBytes, messageOffset, messageLength);
            return byteArray;
        }

        public override void WriteMessage(Message message, Stream stream)
        {
            XmlWriter writer = XmlWriter.Create(stream, this.writerSettings);
            message.WriteMessage(writer);
            writer.Close();
        }
    }

}

While that is all the code I ended up using.  There is one more class I wanted to share for completeness.  During my research, at one point I thought my problem was stemming from passing in username and password security credentials in the header without a nonce and time stamp.

This is a feature Microsoft apparently purposefully excluded because they felt it was less secure and should not be promoted.  Fortunately I ended up not needing it, but I feel for those who do need it and must deal with Microsoft taking the ability away from them because they think they know better than the developer.

Here is the class I ended up using successfully to fix this particular issue.  I copied it almost verbatim from Rick Strahl's Blog.

Custom Credentials Classes

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.ServiceModel.Security;
using System.IdentityModel.Tokens;
using System.Security.Cryptography;

namespace Services
{
    public class WsNonceCustomCredentials : ClientCredentials
    {
        public WsNonceCustomCredentials() { }

        protected WsNonceCustomCredentials(WsNonceCustomCredentials cc) : base(cc) { }

        public override System.IdentityModel.Selectors.SecurityTokenManager CreateSecurityTokenManager()
        {
            return new WsNonceCustomSecurityTokenManager(this);
        }

        protected override ClientCredentials CloneCore()
        {
            return new WsNonceCustomCredentials(this);
        }
    }

    public class WsNonceCustomSecurityTokenManager : ClientCredentialsSecurityTokenManager
    {
        public WsNonceCustomSecurityTokenManager(WsNonceCustomCredentials cred) : base(cred) { }

        public override System.IdentityModel.Selectors.SecurityTokenSerializer CreateSecurityTokenSerializer(System.IdentityModel.Selectors.SecurityTokenVersion version)
        {
            return new WsNonceCustomTokenSerializer(System.ServiceModel.Security.SecurityVersion.WSSecurity11);
        }
    }

    public class WsNonceCustomTokenSerializer : WSSecurityTokenSerializer
    {
        public WsNonceCustomTokenSerializer(SecurityVersion sv) : base(sv) { }

        protected override void WriteTokenCore(System.Xml.XmlWriter writer,
                                                System.IdentityModel.Tokens.SecurityToken token)
        {
            UserNameSecurityToken userToken = token as UserNameSecurityToken;
            if (userToken == null) return;

            string tokennamespace = "o";

            DateTime created = DateTime.Now;
            string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ");

            // unique Nonce value - encode with SHA-1 for 'randomness'
            // in theory the nonce could just be the GUID by itself
            string phrase = Guid.NewGuid().ToString();
            var nonce = GetSHA1String(phrase);

            // in this case password is plain text
            // for digest mode password needs to be encoded as:
            // PasswordAsDigest = Base64(SHA-1(Nonce + Created + Password))
            // and profile needs to change to
            //string password = GetSHA1String(nonce + createdStr + userToken.Password);

            string password = userToken.Password;

            writer.WriteRaw(string.Format(
            "<{0}:UsernameToken u:Id=\"" + token.Id +
            "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" +
            "<{0}:Username>" + userToken.UserName + "</{0}:Username>" +
            "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\">" +
            password + "</{0}:Password>" +
            "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" +
            nonce + "</{0}:Nonce>" +
            "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace));
        }

        protected string GetSHA1String(string phrase)
        {
            SHA1CryptoServiceProvider sha1Hasher = new SHA1CryptoServiceProvider();
            byte[] hashedDataBytes = sha1Hasher.ComputeHash(Encoding.UTF8.GetBytes(phrase));
            return Convert.ToBase64String(hashedDataBytes);
        }

    }
}

My final assessment of the situation is that Microsoft really screwed things up by removing this feature from WCF.  However, with a lot of work, it is possible to hack the change back in.  Hopefully Microsoft will fix this problem some day.  Whether it is an oversight or a feature on their part, the decision they made was a bad one for those of us actually having to do the work to undo what they did.

NetTeirs and Reflection

By now NetTiers is a rather aging templating system.  Many people have complained that it is not being actively maintained and that there are a lot of errors in it, and to some degree they are correct.

NetTiers gets random updates from those using it, but not very frequently.  The code is definitely not bug free, however it is not chalk full of bugs either.  It does rely on Microsoft's EnterpriseLibrary sweet, which is no longer maintained; although the documentation claims this is not a requirement.

But NetTiers has several key features and functionality that had me pick it up after testing multiple data access layer solutions.

• It generates a direct copy of your database into C# code with just the click of a button.  Very few other data access layers can claim that, and their solutions do not feel as smooth as the Codesmith platform that NetTiers runs on.
•  During the generation process NetTiers standardizes the table and column names found in your database, so they all start capitalized, words it identifies are capitalized, and table prefixes can be optionally stripped off.
• It uses meta data from sql regarding indexes and foreign keys to auto generate methods to get data out of the database.
• That meta data also allows links to be made between objects in the generated c# code so you can load and reference foreign key relationships easily.
• Stored Procedures can be auto generated for those who want the added security and to avoid dynamic sql.
• It has a reasonably powerful parameterized sql engine when required.
• A caching layer allows the developer to cash most result sets that come back from the database without additional work.  Which can be useful for small static tables.
• It picks up custom stored procedures and adds methods onto the generated objects to allow the developer to call them without any additional effort.
• There are quite a few template options to allow the developer to customize the resulting code to their environment.

Many data access layers claim features similar to these, although oftentimes with less control over them.  However, there are three features that make NetTiers unique and keep it valuable and relevant:

The direct copy of the database into standard c# code takes a huge load off of the developer who no longer has to write all that plumbing code.  The generated code is in the form of normal C# objects that match the names in your database.  This means there is no guesswork as to what an object name is going to be.  It also means full object oriented and intellisense capable access to every object in the database.

Generated code is compile time safe; so most errors in the database will show up right away and not allow the code to compile.  This code has been used by many people lots of times, so most major bugs have been worked out already.  Having it all generated the same way every time nearly eliminates the ability for a random error to sneak into the data access layer.

One of my favorite features is actually a byproduct of having the data access layer so standardized.  The ability to use reflection heavily.  Because there are so many standards built into nettiers, you can always find the correct get or save methods or primary keys regardless of the object you are working with.  The generated code in these areas all follows the same pattern.

While not perfect ( I have submitted a few corrections back to the code base myself ), the ability to auto generate an entire DAL without effort, and the ability to rely heavily on reflection when using these objects, has made me a believer in NetTiers for many of the projects I work on.

It is not the only solution available.  But for server based C# projects it definitely does shine as a way to drastically reduce development time.

WanCam wireless camera HW0036

I made a mistake in getting this camera.  Despite it having the features it advertised, this camera is missing an RJ45 jack.  For security reasons I never buy a camera without a wired connection.

This camera does work correctly, however they made a couple of annoying design decisions when making this model.  First, the http port by default is 81 rather than the standard 80.  Second they also deviated from the standard OnVif ports and choose to use 10080 for this service; and it is not configurable.

It does not support https, which I consider a negative, although not horrible as I rarely use the web interface once configured.

Fortunately you can change the http port, and you can change the default password.  The camera works very well with Milestone.  As long as you only want wireless then this is a good camera, however because of that reason I would never recommend, or purchase another one.

EDIT: Within a week of owning this camera it began to have issues turning on.  A few days later the problem was so bad the camera no longer functioned.  It is possible that I simply got a lemon, or this particular model has a bad design flaw.  I was able to get a partial refund, threw the camera away, and will never get another of this model.

KanKun Wireless Plug

I recently purchased  a wireless KanKun plug, specifically because I heard that it was running a version of OpenWRT and could easily be modified.

I found the primary directions here.

There were a couple of challenges.  The first was getting the plug on the wireless network.  The Chinese directions and app did not function at all, and I tried them multiple times.

The good news is that the plug successfully resets all failed attempts very well with its reset process.  Eventually I was successful following these steps:

- Use a laptop to connect to the wireless network the plug defaults to creating.
- Lookup the Plugs IP based on the DHCP address assigned to the laptop because it did not create the network that most examples were claiming it would.
- Use Putty to SSH into the plug with a username of 'root'.  My default password was p9z34c, however others have claimed it might be admin or 1234.
- Edit the following file with 'vi': /etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '9'
        option hwmode '11ng'
        option path 'platform/ar933x_wmac'
        list ht_capab 'SHORT-GI-20'
        list ht_capab 'SHORT-GI-40'
        list ht_capab 'RX-STBC1'
        list ht_capab 'DSSS_CCK-40'
        option htmode 'HT20'
        option disabled '0'
        option country 'CN'

config wifi-iface
        option device 'radio0'
        option network 'wwan'
        option ssid 'MyWirelessSID'
        option encryption 'psk2'
        option mode 'sta'
        option key 'MyWirelessPassword'

I made sure the channel was set correctly since I run my wireless network on 9 rather than the standard 11.  I also struggled a bit with the encryption since I run WPA2 Personal; I finally got the very logical instructions to set encryption to 'psk2'.

- The second critical file to edit is: /etc/config/network

config interface 'loopback'

        option ifname 'lo'

        option proto 'static'

        option ipaddr '127.0.0.1'

        option netmask '255.0.0.0'

config globals 'globals'

        option ula_prefix 'fd59:45ed:8ead:0000:/48'

config interface 'lan'

        option ifname 'eth0'

        option force_link '1'

        option type 'bridge'

        option proto 'static'

        option ipaddr '192.168.145.253'

        option netmask '255.255.255.0'

        option ip6assign '60'

config interface 'wan'

        option proto 'dhcp'

        option ifname 'eth1'

config interface 'wwan'

        option proto 'static'

        option ipaddr '192.168.1.32'

        option gateway '192.168.1.1'

        option netmask '255.255.255.0'

I modified the MAC address in this file, adding the '0000', however leaving it empty would probably have been fine as well from what others have said; and considering it was working prior.  Adding the last 'config interface wwan' section is the most critical.  It is used by the wireless file for interface settings and specifies the static IP network necessary to communicate on this network.

- 'reboot' the plug

- With the plug now successfully connecting to my network I was able to return to my main computer for further configuration.

- My next step was to copy in a simple CGI script someone created to allow me to remotely control the plug.  I did have to modify the following script slightly because my RELAY_CTRL was different from the ones they were using.

- I had to create the folder /www/cgi-bin

- And create the following file: /www/cgi-bin/relay.cgi

#!/bin/sh

echo "Content-Type: text/plain"

echo "Cache-Control: no-cache, must-revalidate"

echo "Expires: Sat, 26 Jul 1997 05:00:00 GMT"

echo

RELAY_CTRL=/sys/class/leds/i-konke\:red\:relay/brightness

case "$QUERY_STRING" in

state) 

case "`cat $RELAY_CTRL`" in

0) echo "OFF"

;;

1) echo "ON"

;;

esac

;;

on) 

echo 1 > $RELAY_CTRL

echo OK

;;

off) 

echo 0 > $RELAY_CTRL

echo OK

;;

esac

- Unfortunately I discovered that the uhttpd service did not exist on my plug.  I had to turn to a very helpful Internet community who was able to give me the required files.  You can download them here.

- Inside the rar file you should see a kkplug folder with the most likely folder structure and the files that go in each folder.  Use WinSCP to copy the files to the plug.
- Run the following command

/etc/init.d/uhttpd enable

- This creates a sym link that causes the plug to execute the startup file on boot.

- 'reboot' again to test and verify that the service will start correctly.

- When the plug comes up you should be able to use a web browser and control the plug in this way.

http://192.168.1.32/cgi-bin/relay.cgi?on

Knewmart Indoor IP Camera

I am always on the lookout for inexpensive yet quality cameras to integrate with my security system.  If you buy a camera today from almost any store it is nearly a given that it will come with an app that you can use to view the camera remotely.  But aside from a few small motion detection perks that is usually about the end of it.  Many cameras do not play well with other cameras, or make it easy and cheap to actually record like a real security system.

For this reason I have come up with the following rules when buying cameras:
1. It must support OnVif.  This is a standards protocol that guarantees the camera will at least try and play nice with most of the security systems out there.
2. It must support H.264 encoding.  This is a compression algorithm that significantly reduces the amount of storage required when recording for long periods.
3. It must have an Ethernet jack.  Many people like the idea of going wireless for convenience, but even wireless cameras need a power cord.  However, wireless has the major downside of being insecure.
  - Not only is it much easier to hack into a wireless feed.
  - But if you put the camera next to your microwave and turn it on you will realize it is a piece of cake to simply overwhelm the camera with radiation, disabling the feed.
  - Also a nice quality camera will be sending so much data over the wireless network it can overwhelm a normal wireless router, effectively rendering your wireless network useless.
  - And even if you decided to give it its own wireless network, you are still cluttering the radio waves which could end up causing various problems for you in the future.
4. It really should allow for a Static IP to be set, it is just easier to manage networks that way.
5. If it is an outdoor camera it must support POE.  Power Over Ethernet allows me to run just one low voltage cable to the camera.  Trying to mess with high voltage extension cords and a power cable just is not worth the trouble.

As a Note.  For outdoor cheap cameras I have had better luck with the few large LED configuration over the numerous small LEDs for night vision.  Cheaper cameras are not made with the same care, and you can end up with the small LEDs not being positioned correctly and washing out your image at night.

Camera technology is moving fast, but after testing quite a few brands I had settled on a really nice outdoor camera that met all my requirements and worked very well indoors also.  But then two new brands came on the scene, Wancam and Knewmart.

Today I am looking at the Knewmart (no model).

Positives
- It fits all my requirements for an indoor camera (eg, it supports OnVif and H.264 and allows for a static IP to be set, and has an Ethernet jack)
- It has wireless built in in case I do ever need it, and the ability to disable it for security if I don't use it.
- For those of us who do not like installing more apps on our phone it has a built in web server so I can configure it from my computer, which I love.  It is the first web interface I have seen on a less than $50 camera that actually works well.
- The web configuration interface is very simple and easy to use and really started me falling in love with this camera immediately.
- The camera ships with DHCP enabled by default, so it auto connects to your wired network when you plug it in.  This is a much better design in my opinion than cameras that try and stand up their own wireless network for configuration.
- It has a default username/password of admin/admin.  It does have a user and guest account, which is nice, but it has no way to disable them, so make sure you change all the passwords.
- For those who are paranoid about configuring their own cameras, it does come with a QR code to auto configure basic connectivity in their native app.
- It has the ability for dual streams at different quality levels, which was unexpected although it is probably becoming more standard in the industry.
- For those who do not run separate NVR software, it has the built in ability to trigger alarms during a time span you schedule and send you an email with a picture of what caused the alarm. However, I did not test this.
- It has little motors so you can pan and tilt it, which is new for the less expensive cameras, and I absolutely love it.  This Knewmart has the widest range of motion I have experienced yet, it Pan: 355°, Tilt: 90°
- It does work with the E-View7 app, which I liked because I already use that app for other cameras.  They also made a special P2PIPC app according to the directions that came with it.
- For those who are not quite ready to built an NVR system, but still want to record some video locally, it does come with a MicroSD card slot.  While this does seem to be a popular feature, it is a feature I have never personally found a good use for.
- It also comes with the ability to talk out of it and hear through it, another set of features I absolutely love in this camera.
- There is port labeled as a headphone jack, although I think the picture is misleading, I believe it is a Microphone jack in case you want better audio reception.  The OnVif api supports this conclusion by listing two microphones.
- I use Milestone's XProtect software, and after enabling OnVif it had no issue connecting to the camera on the first try. Microphone and pan/tilt work great.
- As far as cameras go, this thing actually looks very nice.  I question whether it is a bit too big and bulky for an indoor camera that you typically want to be unobtrusive; but the smooth fluids lines on it help make it a more attractive decoration.  Plus there is the argument that being bigger means better quality.
- It has the built-in ability to contact time.windows.com to set it's own time and handle daylight savings time; you just have to choose the right time zone.  This is a very handy feature that is often overlooked.
- It has the standard Flip and Mirror options for the picture, which are really nice when you want to mount the camera in an odd place, like the ceiling.
- I will give this company credit for effort in support also, this is a nice in-expensive camera, and they respond to support requests.  It may not be American level support, but they also do not leave you hanging which is really nice.

- It does have a reset button, I mention this obvious feature only because I have actually seen cameras without one.
- It seamlessly switches between wireless and wired without requiring a hard reboot.

- And finally it takes a unique approach to wall and ceiling mounting.  Most cameras like this have holes in their bottom you just slide over screws.  This camera ships with a pretty configurable mounting arm.  While I am not sure the aesthetics of it are great, it certainly does allow for extreme flexibility in how and where you mount the camera.

Negatives
- The directions that came with the camera are very basic and intended for a user who only cares about getting the camera running.  If you want to do anything advanced with it you will probably need to use the web interface on your computer and do a little techie research if you do not already understand some of the terminology.
- It does have a small glitch/feature where you have to manually unplug it after making some change to the OnVif settings to actually make OnVif work, it comes enabled by default it just does not work by default.  I have experienced this with the wancam as well.

- I was disappointed that the speaker does not work over OnVif with my XProtect NVR system, but I expected that from other similar cameras I have had.  Hopefully some day they add that feature.
- It does not support POE.  But for an indoor camera that is not a deal breaker for me.

- Like other cameras of this type, when you first turn it on it tests its range of motion, likely to configure the limits in the software.  It has always concerned me that the camera trying to go past its limit during these tests will eventually burn out the motor, but so far that has not happened.
- It does not support Https, however for those who are that paranoid about security (like me) they are probably running their cameras on a closed internal secure network anyway, so it would not be much of an issue.
- If you tilt the camera too far down then its night vision LEDs reflect on itself causing the image to blur white.

- The default IRCut value seems to be too low, in low light the camera starts flickering the IR on and off.  You can either fiddle with the settings, or just switch it to manual mode.
- It is not very easy to unplug a network cable once plugged into it, especially with bigger fingers.
- When used via OnVif with the XProtect software, there is an error in the OnVif protocol when used over wireless that causes a lag in the image frames triggering XProtect to constantly report it as disconnected.  Oddly enough, you can get around this error by plugging it into a network cable, waiting for the transition, then unplugging the network cable.  It will then work correctly for awhile.

All in all, this is a great camera that can support a range of users from those starting out, to those who are quite a ways along in building a system.  If it were not for the larger size (roughly double it's WanCam counterpart) it would become my go-to indoor camera for home systems.  However, it beats out other inexpensive cameras in so many areas that it is still going to be at the top of my list for consideration when I recommend cameras to people