Friday, May 11, 2018

Pacific Power Hourly Energy Savings

On the North West coast of the United States is a company called Pacific Power.  They are the government approved power provider.

There is a special program they have that purports to incentivize people to shift their power usage to off peak periods.  This is a very good idea to help balance out the power grid, and I am a huge fan of the idea.  However, it does cause a great deal of work for those who shift their usage and they do deserve to be incentivized to do so.

Unfortunately, it turns out that the program Pacific Power runs is a scam.  Their documentation, and their representatives who try and explain the documentation, claims that the normal household is charged 3 cents per kilowatt hour.  Once switched to the new program the household will be charged 6 cents per kilowatt hour for on peak usage, and only 1 cent per kilowatt hour for off peak usage.  This seems reasonable, and a hard working person should be able to save some money by changing their habits fairly drastically.
( One of the documents actually says it will be a negative 1 cent, the reps just seem to explain it poorly )

The reality of this program is far different from what is advertised.

In reality, Pacific Power does not stop charging the household for anything, they continue charging the standard electrical fees, and then they tack on an ADDITIONAL 6 cents per kilowatt hour for peak usage, and a -1 cent per kilowatt hour for off peak usage.

So, by completely revamping your lifestyle, you might save $1-2 a month if you are lucky, if you slip then you will suddenly be paying a massively inflated bill on top of the normal electrical bill you were already paying.

Friday, March 16, 2018

Microsoft TFS Server Custom SQL Query

Microsoft's TFS 2017 has come a long ways, but is still a pretty terrible product.  That said, it is still the best option for many things as a Microsoft developer.

One thing I have seen a lot of people on the Internet asking about is various ways to get at the TFS information stored in SQL.  Microsoft obviously refuses to answer these questions just pointing people to their APIs.  However, sometimes it is just easier to run a quick SQL query.

For those people who are just wondering where Microsoft is hiding all the various fields, I have spent a good bit of time tracking some of them down.  Remember this code is for SQL 2017 and is unlikely to work for other versions, but it might point you in the right direction.  All this is is one of my queries with some comments, I will leave it up to you to go look at the tables and see how I use them to get the data.

The very last join I do is for a custom HTML field that I created in TFS, a very similar join on that table could be used to get other HTML fields as well, such as comments or description fields.
// Dashboards.tbl_Widget stores the Dashboard settings, you can hack changes here
// dbo.QueryItems has the settings for each query
// tbl_ChangeSet stores the change sets
// vw_WorkItemComments stores all the work item comments, including comments generated when changesets are submitted
// tbl_Version holds actual file references, can be linked to changesets using VersionFrom column
// WorkItemFiles are links between workitems and the changesets
// dbo.WorkItemLongTexts stores the long texts from bugs, like the main body
// dbo.tbl_TagDefinition contains the tag names themselves joins dbo.tbl_PropertyDefinition.Name on TagId
// dbo.tbl_Field contains the descriptions for the FieldId's from tbl_WorkItemCustomLatest
// dbo.tbl_PropertyDefinition string guid table join on dbo.tbl_PropertyValue on PropertyId integer value, ArtifactId is lookup key which is WorkItemId number converted to hex

select [System.Id] ID, l_title.[TextValue] [Title], l_approvedby.[StringValue] [ApprovedBy], l_valuearea.[StringValue] [ValueArea], l_priority.[IntValue] [Priority], l_waitingon.StringValue [WaitingOn]
-- Sub Select for Tags
,(select substring((select distinct ', ' + td.[Name]
from dbo.tbl_PropertyValue pv
Inner Join dbo.tbl_PropertyDefinition pd on pv.PropertyId = pd.PropertyId
Inner Join dbo.tbl_TagDefinition td on pd.[Name] = 'Microsoft.TeamFoundation.Tagging.TagDefinition.' + cast(td.TagId as varchar(100))
    Where pv.InternalKindId = 16 And pv.ArtifactId = CONVERT(VARBINARY(8), l.[System.Id])
for xml path('')), 3, 9000)) as Tags
,l_deploy.Words DeployNotes
from vw_denorm_WorkItemCoreLatest l
    inner join vw_denorm_WorkItemCustomLatest l_title on l.[System.Id] = l_title.Id And l_title.FieldId = 1
    left join vw_denorm_WorkItemCustomLatest l_approvedby on l.[System.Id] = l_approvedby.Id And l_approvedby.FieldId = 10133
left join vw_denorm_WorkItemCustomLatest l_valuearea on l.[System.Id] = l_valuearea.Id And l_valuearea.FieldId = 10055
    left join vw_denorm_WorkItemCustomLatest l_priority on l.[System.Id] = l_priority.Id And l_priority.FieldId = 10029
    left join vw_denorm_WorkItemCustomLatest l_waitingon on l.[System.Id] = l_waitingon.Id And l_waitingon.FieldId = 10134
    left join (select Max(AddedDate) AddedDate, Id from WorkItemLongTexts Where FldID = 11135 Group By Id ) l_deploy_grp on l.[System.Id] = l_deploy_grp.Id
left join WorkItemLongTexts l_deploy on l.[System.Id] = l_deploy.Id And l_deploy.FldID = 11135 And l_deploy_grp.AddedDate = l_deploy.AddedDate
Where
l.[System.IsDeleted] = 0
    And l.[System.State] = '20 Testing'
    And l.[System.AreaPath] = '\VerizonRAB'
And ( l.[System.WorkItemType] = 'Feature' Or l.[System.WorkItemType] = 'Bug' Or l.[System.WorkItemType] = 'Task' )



Wednesday, January 3, 2018

WatchGuard FireBox Phone VPN Configuration

We recently migrated from a very old Cisco ASA firewall to a couple of WatchGuard FireBox M200 devices.  So far my assessment is that they are very capable devices with lots of features, but inadequately documented and the company has poor support.

Part of my negative experience in regards to support may be from buying through a third party re-seller.  WatchGuard does not want to talk to you directly, they want to talk to your re-seller.

Most of the configuration and setup went fairly smoothly.  Due to poor documentation there were some gotcha's when setting up active directory authentication in regards to case sensitivity.  The little documentation WatchGuard does have is well formatted and reads nicely.  Unfortunately, due to there being so little of it you end up trying to reference the wrong documentation to help fill in the blanks for missing documentation.  Many companies try and fill this gap by providing help forums that they monitor and respond to questions on which then fills in all the missing information.  WatchGuard seems to be missing a good implementation of this, and they do not seem to respond when people reach out to them.

You can see a perfect example of all of WatchGuard's issues by looking at their android vpn app in the Google Play store.  The app is poorly written, functions horribly, and they ignore all the people providing negative feedback who are essentially begging for help.

Towards the end of my own WatchGuard configuration I intended to setup an IPSec configuration for phones to connect to as indicated in their documentation.  Unfortunately their own phone vpn client is utterly worthless leaving you to try and find something else.

I did use WatchGuard's iPhone configuration instructions to get the native Android VPN client to connect.  Unfortunately it ended up having two major issues that caused me to dump it.  The first is that WatchGuard has a limitation or perhaps a bug in it's IPSec implementation for Active Directory authentication.  As near as I can tell, either SSL or IPSec can authenticate via Active Directory but not both.  It is possible that by deleting the SSL config, setting up IPSec, then re-creating the SSL config I could get around this bug, but it was not worth attempting at the moment.  When using AD authentication in the configuration I had setup, it would connect for a few seconds, then disconnect.  Which tells me there must me some sort of timing mismatch.  You would think that would be simple to adjust and overcome, but again the lack of documentation means I either have to pay for support or just deal with it for now.

The second problem, is that even when using the WatchGuard database native authentication, the native client on android does not seem to hold in a connection when the phone goes to sleep.  So if I set my phone down for a few minutes and come back to it I have to re-connect every time.

Fortunately WatchGuard does support OpenVPN.  So I was able to download a third party client that works with the existing SSL configuration successfully.  Not ideal, but it works for now.

- 1: download the client.ovpn attachment to your phone.  You can find this by going to the ip address of your firewall in a browser, authenticate, and download the "Mobile VPN with SSL client profile".
- 2: Open up the App store on your respective device and find the app OpenVPN Connect (not just OpenVPN, it has to say Connect).
    o Install the app and open it.
- 3: Click on the menu in the upper right, the three dots.
- 4: Click the menu item Import Profile from SD card.
- 5: navigate to the folder containing the client.ovpn attachment that you downloaded in step 1.  For me it was in my Download folder.
- 6: Click on the client.ovpn file and click the Select button.  It may be confusing because the client.ovpn line you selected will not highlight.
- 7: If you chose the correct file you should now see an IP address in the OpenVPN Profile area.
    o Enter your Active Directory username and password into the provided fields.
    o Click Connect

Monday, December 11, 2017

Controling SOAP security header “mustUnderstand” attribute in WCF client

I was recent tasked with connecting .NET WCF with an older java soap service.  Unfortunately the service was not standard and needed more WCF customization than normal, starting with the security.
I needed to include both a client x509 certificate for the Transport tunnel, and a username/password combination in the soap header itself.  By playing around with the web.config binding configurations I could have eventually achieved this.  However that was not the end of the customizations it needed.
This service needed the EncodedMustUnderstand flag set to false for the security header.  Back with WSE that used to be a very simple flag to set, but for some reason Microsoft decided to remove that capability from WCF.  One guy claimed to find a simple fix for this, however his fix did not work for me.
Throughout the years since WCF has been introduced many people have had this or very similar issues when dealing with Java based SOAP services.  I have yet to find someone who has discovered a good solution.  In my own two week journey to solving this issue I had to do some pretty serious customization to WCF.  By sharing my discoveries here I hope to help other people shorted the time they spend enduring the same pain.
The first thing I did was to create an extension method for the service allowing me to create a CustomBinding for it.  This method served as the hub to which I attached various other classes and methods as I built them.  It seems daunting, and it really is.  Separating out the various ideas into their own files was the approach I took to keeping the major areas of functionality separate in my head.

Base Helper Class
using System;
using System.Configuration;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Security;
using System.ServiceModel.Security.Tokens;

namespace Services
{
    public static class ClientHelper
    {

        public static PClient Initialize(this PClient client)
        {
            var urlConfig = ConfigurationManager.AppSettings["Url"].ToString();

            if (string.IsNullOrEmpty(urlConfig))
                throw new Exception("Missing Url Config");

            var url = new UrlSecurityConfig(urlConfig);

            //System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;

            var security = SecurityBindingElement.CreateCertificateOverTransportBindingElement();
            security.IncludeTimestamp = true;
            security.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic256;
            security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10;
            security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
            security.SecurityHeaderLayout = SecurityHeaderLayout.Lax;
            security.EnableUnsecuredResponse = true;

            //security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11;
            //security.EndpointSupportingTokenParameters.SignedEncrypted.Add(new X509SecurityTokenParameters(X509KeyIdentifierClauseType.Any, SecurityTokenInclusionMode.AlwaysToRecipient)); // add specific x509 cert security            security.EndpointSupportingTokenParameters.Signed.Add(new UserNameSecurityTokenParameters()); // add specific username security feature
            //security.DefaultAlgorithmSuite = new Basic128Sha256Rsa15Sha1AlgorithmSuite(); // when we need to tweak the security suite


            //var encoding = new TextMessageEncodingBindingElement();
            //encoding.MessageVersion = MessageVersion.CreateVersion(EnvelopeVersion.Soap11, AddressingVersion.None);// MessageVersion.Soap11;
            ////encoding.MessageVersion = MessageVersion.Soap11WSAddressingAugust2004;
            //encoding.WriteEncoding = Encoding.UTF8;


            var transport = new HttpsTransportBindingElement();
            transport.MaxReceivedMessageSize = 20000000; // 20 megs
            transport.RequireClientCertificate = false;
            transport.TransferMode = TransferMode.Buffered;
            transport.DecompressionEnabled = false;

            CustomBinding binding = new CustomBinding();
            binding.Elements.Add(security);
            //binding.Elements.Add(encoding); // add normal encoding
            binding.Elements.Add(new MustUnderstandOffBindingElement("UTF-8", "text/xml", MessageVersion.Soap11)); // add custom encoder
            binding.Elements.Add(transport);

            var x509Config = ConfigurationManager.AppSettings["x509"].ToString();
            var X509Cert = Common.Get509Cert(x509Config);

            // THIS IS WHEN THE CERT NEEDS A SPECIFIC DOMAIN IDENTITY SPECIFIED
            //var identity = EndpointIdentity.CreateX509CertificateIdentity(X509Cert);
            //var address = new EndpointAddress(new Uri(url), identity);
            ////var binding = new WSHttpBinding(SecurityMode.Message);
            ////binding.Security.Message.ClientCredentialType = MessageCredentialType.Certificate;
            //var factory = new ChannelFactory<CoAServices_v1r21_P>(binding, address);
            ////factory.Endpoint.EndpointBehaviors.Remove(typeof(System.ServiceModel.Description.ClientCredentials));
            ////factory.Endpoint.EndpointBehaviors.Add(new Services.WsNonceCustomCredentials());
            //factory.Credentials.UserName.UserName = username;
            //factory.Credentials.UserName.Password = password;
            //factory.Credentials.ClientCertificate.Certificate = X509Cert;
            //factory.Credentials.ServiceCertificate.DefaultCertificate = X509Cert;

            client.Endpoint.Binding = binding;
            client.Endpoint.Address = new EndpointAddress(url.Url);
            //client.Endpoint.EndpointBehaviors.Add(new ExClientBehavior()); // adding a custom behavior built in a custom class
            //client.Endpoint.EndpointBehaviors.Add(new MustUnderstandBehavior(false)); // create a behavior with the MustUnderstand attribute set to false, does not work for headers

            //// replace normal credencials with custom ones that generate Nonce:
            //client.ChannelFactory.Endpoint.Behaviors.Remove<System.ServiceModel.Description.ClientCredentials>();
            //client.ChannelFactory.Endpoint.Behaviors.Add(new Services.WsNonceCustomCredentials());

            client.ClientCredentials.UserName.UserName = url.UserName;
            client.ClientCredentials.UserName.Password = url.Password;

            client.ClientCredentials.ClientCertificate.Certificate = X509Cert;
            //client.ClientCredentials.ServiceCertificate.DefaultCertificate = X509Cert;
            //client.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.None;
            //client.ClientCredentials.ServiceCertificate.Authentication.RevocationMode = X509RevocationMode.NoCheck;

            return client;
            //return factory.CreateChannel();
        }

    }


}


As you can see, there is lots of commented out code to plug in various pieces of functionality.  I ended up not needing half of it, but there was no point in throwing away code that functioned.  For the purposes of this post, only the commented out ExClientBehavior lines and the custom Nonce credencials will be discussed in addition to the working lines of code.

One of the first steps I took was to create a helper class with a method to get my x509 certificate.  I later added a method to do some basic url config parsing.  This ended up being the easiest part of the whole process.

EDIT: A quick side note, when generating the code below for this blog post, it kept wanting to rename X509Certificate2Collection and other class names as 17bH1SYLoBdGsBaDedPR2EE3JUt8oRS7qd.  A very odd quirk, but I have seen that weird name in several Microsoft posts as well, so apparently I am not the only one who has had that issue when writing posts. 

EDIT2: It turns out the weird renaming was caused by a Chrome extension I am running called uBlock.  So if you see something similar, then check your extensions.  In my case, white listing the blogger domain fixed the issue.

Common Helper Methods
using System;
using System.Linq;
using System.Security.Cryptography.X509Certificates;

namespace Services
{
    public static class Common
    {
        public static X509Certificate2 Get509Cert(string LocationConfig)
        {
            var locationParts = LocationConfig.Split(':');
            if (locationParts.Length < 2)
                throw new Exception("Invalid X509 Config");

            X509Certificate2 cer = new X509Certificate2();
            X509Store store = locationParts[0] == "CurrentUser" ? new X509Store(StoreLocation.CurrentUser) : new X509Store(StoreLocation.LocalMachine);

            store.Open(OpenFlags.ReadOnly);

            X509Certificate2Collection cers = store.Certificates.Find(X509FindType.FindBySubjectName, locationParts[1], false);
            if (cers.Count == 0)
            {
                throw new Exception("Can't find X509 Cert.");
            }

            var cert = cers[0];

            store.Close();
            return cert;
        }
    }

    public class UrlSecurityConfig
    {
        public string Url { get; protected set; }
        public string UserName { get; protected set; }
        public string Password { get; protected set; }

        public UrlSecurityConfig(string config)
        {
            if(!config.Contains('@'))
            {
                Url = config;
                return;
            }

            var uendStart = config.LastIndexOf('@');
            var uStart = config.Substring(0, config.IndexOf('/') + 2);
            var uEnd = config.Substring(uendStart + 1);
            Url = uStart + uEnd;

            var nameEnd = config.IndexOf(':', uStart.Length + 1);
            UserName = config.Substring(uStart.Length, nameEnd - uStart.Length);
            Password = config.Substring(nameEnd + 1, uendStart - nameEnd - 1);
        }
    }
}

After trying all sorts of ways to get into the security header using existing settings, I finally decided to write my own custom Behavior.  This code I copied directly from someone else's post who was struggling with something very similar.  Unfortunately, I got a rude shock when I found out that the security headers are not part of the request sent to the behavior.

Without access to those I was unable to modify them, although in testing I discovered that I was unable to affect the output of the headers here anyway despite the code looking like it was working correctly when I stepped through it.  Perhaps I still have something slightly off.

Custom Behavior Classes
using System;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.ServiceModel.Configuration;

namespace Services
{
    public class ExClientBehavior : IEndpointBehavior
    {
        #region IEndpointBehavior Members

        public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
        {
            // no op
        }

        public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
        {
            ExInspector inspector = new ExInspector();
            clientRuntime.MessageInspectors.Add(inspector);
            //clientRuntime.ValidateMustUnderstand = false;

        }

        public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
        {

            // no op
        }

        public void Validate(ServiceEndpoint endpoint)
        {
            // no op
        }

        #endregion
    }


    public class ExClientBehaviorExtensionElement : BehaviorExtensionElement
    {
        public override Type BehaviorType
        {
            get { return typeof(ExClientBehavior); }
        }

        protected override object CreateBehavior()
        {
            return new ExClientBehavior();
        }
    }

    public class ExInspector : IClientMessageInspector
    {

        #region IClientMessageInspector Members

        public void AfterReceiveReply(ref Message reply, object correlationState)
        {
            // no op
            return;
        }

        public object BeforeSendRequest(ref Message request, IClientChannel channel)
        {
            MessageBuffer buffer = request.CreateBufferedCopy(int.MaxValue);

            Message newMessage = buffer.CreateMessage();

            while (newMessage.Headers.Count > 0)
                newMessage.Headers.RemoveAt(0);
            
            foreach (var info in request.Headers)
            {
                newMessage.Headers.Add(MessageHeader.CreateHeader
                    (
                        info.Name,
                        info.Namespace,
                        string.Empty,
                        false,
                        string.Empty,
                        info.Relay
                    )
                );
            }


            request = newMessage;

            return null;
        }

        #endregion
    }
}

My next approach was to try writing my own Custom Encoder.  Many people had stated that was the way to get at the stream both as it was heading out the door, and as it was coming back in before it hit the service code.  I copied Microsoft's code and directions from here, here, and here.

I immediately started running into problems due to that class name rewrite in Microsoft's web posts that I mentioned earlier.  It took me a little while to create names that all worked and referenced each other correctly.

I could have handled the pain it had taken to get this far, but it wasn't over yet.  Microsoft's code has a flaw in it that resulted in a loop which consistently generated stack overflow errors.  I later discovered that the error centered around the "this.factory = factory;" line.  It generated an internal reference to the parent object that the parent then referenced causing a recursive infinite loop.

If there had been any sort of decent error that would have been easy to find and work through.  Unfortunately, these were the errors and Inner Exceptions that it kept generating:

Message: An error occurred while making the HTTP request to https://service. This could be due to the fact that the server certificate is not configured properly with HTTP.SYS in the HTTPS case. This could also be caused by a mismatch of the security binding between the client and the server.

Message: The underlying connection was closed: An unexpected error occurred on a send.

Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Stepping through the code did show me that a stack overflow was occurring, but it was not obvious that it was a recursive reference that was causing it.  And, since my code had been copied from an official Microsoft website, my first instinct was to trust it and look elsewhere.  So I spent a long time looking at the exceptions and trying to figure out what they could mean.

During this time I spent a lot of time in Fiddler working with the returning soap message and tweaking it to see how it would affect .net.  I discovered that it is actually possible to strip off the characters surrounding the soap envelope that trigger the HTTP.SYS error in .net.  This allowed me to test the service even when sending from an HTTP source rather than the required HTTPS source.

I finally ended up with this working code.  You will notice that the primary difference from Microsoft's classes is the replacement for the "mustUnderstand" line, which was the hack I put in to toggle that flag as the message went out the door.

Custom Encoder Classes
using System;
using System.Text;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.IO;
using System.Xml;

namespace Services
{
    public class MustUnderstandOffBindingElement : MustUnderstandOffBindingElement, IWsdlExportExtension
    {
        private MessageVersion msgVersion;
        private string mediaType;
        private string encoding;
        private XmlDictionaryReaderQuotas readerQuotas;

        MustUnderstandOffBindingElement(MustUnderstandOffBindingElement binding)
            : this(binding.Encoding, binding.MediaType, binding.MessageVersion)
        {
            this.readerQuotas = new XmlDictionaryReaderQuotas();
            binding.ReaderQuotas.CopyTo(this.readerQuotas);
        }

        public MustUnderstandOffBindingElement(string encoding, string mediaType,
            MessageVersion msgVersion)
        {
            if (encoding == null)
                throw new ArgumentNullException("encoding");

            if (mediaType == null)
                throw new ArgumentNullException("mediaType");

            if (msgVersion == null)
                throw new ArgumentNullException("msgVersion");

            this.msgVersion = msgVersion;
            this.mediaType = mediaType;
            this.encoding = encoding;
            this.readerQuotas = new XmlDictionaryReaderQuotas();
        }

        public MustUnderstandOffBindingElement(string encoding, string mediaType)
            : this(encoding, mediaType, MessageVersion.Soap11WSAddressing10)
        {
        }

        public MustUnderstandOffBindingElement(string encoding)
            : this(encoding, "text/xml")
        {

        }

        public MustUnderstandOffBindingElement()
            : this("UTF-8")
        {
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return this.msgVersion;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.msgVersion = value;
            }
        }


        public string MediaType
        {
            get
            {
                return this.mediaType;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.mediaType = value;
            }
        }

        public string Encoding
        {
            get
            {
                return this.encoding;
            }

            set
            {
                if (value == null)
                    throw new ArgumentNullException("value");
                this.encoding = value;
            }
        }

        // This encoder does not enforces any quotas for the unsecure messages. The 
        // quotas are enforced for the secure portions of messages when this encoder
        // is used in a binding that is configured with security. 
        public XmlDictionaryReaderQuotas ReaderQuotas
        {
            get
            {
                return this.readerQuotas;
            }
        }

        #region IMessageEncodingBindingElement Members
        public override MessageEncoderFactory CreateMessageEncoderFactory()
        {
            return new MustUnderstandOffEncoderFactory(this.MediaType,
                this.Encoding, this.MessageVersion);
        }

        #endregion


        public override BindingElement Clone()
        {
            return new MustUnderstandOffBindingElement(this);
        }

        public override IChannelFactory<TChannel> BuildChannelFactory<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.BuildInnerChannelFactory<TChannel>();
        }

        public override bool CanBuildChannelFactory<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            return context.CanBuildInnerChannelFactory<TChannel>();
        }

        public override IChannelListener<TChannel> BuildChannelListener<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.BuildInnerChannelListener<TChannel>();
        }

        public override bool CanBuildChannelListener<TChannel>(BindingContext context)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            context.BindingParameters.Add(this);
            return context.CanBuildInnerChannelListener<TChannel>();
        }

        public override T GetProperty<T>(BindingContext context)
        {
            if (typeof(T) == typeof(XmlDictionaryReaderQuotas))
            {
                return (T)(object)this.readerQuotas;
            }
            else
            {
                return base.GetProperty<T>(context);
            }
        }

        #region IWsdlExportExtension Members

        void IWsdlExportExtension.ExportContract(WsdlExporter exporter, WsdlContractConversionContext context)
        {
        }

        void IWsdlExportExtension.ExportEndpoint(WsdlExporter exporter, WsdlContractConversionContext context)
        {
            // The MessageEncodingBindingElement is responsible for ensuring that the WSDL has the correct
            // SOAP version. We can delegate to the WCF implementation of TextMessageEncodingBindingElement for this.
            TextMessageEncodingBindingElement mebe = new TextMessageEncodingBindingElement();
            mebe.MessageVersion = this.msgVersion;
            ((IWsdlExportExtension)mebe).ExportEndpoint(exporter, context);
        }

        #endregion
    }

    public class MustUnderstandOffEncoderFactory : MessageEncoderFactory
    {
        private MessageEncoder encoder;
        private MessageVersion version;
        private string mediaType;
        private string charSet;

        internal MustUnderstandOffEncoderFactory(string mediaType, string charSet, MessageVersion version)
        {
            this.version = version;
            this.mediaType = mediaType;
            this.charSet = charSet;
            this.encoder = new MustUnderstandOffEncoder(this);

        }

        public override MessageEncoder Encoder
        {
            get
            {
                return this.encoder;
            }
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return this.version;
            }
        }

        internal string MediaType
        {
            get
            {
                return this.mediaType;
            }
        }

        internal string CharSet
        {
            get
            {
                return this.charSet;
            }
        }
    }

    public class MustUnderstandOffEncoder : MessageEncoder
    {
        private XmlWriterSettings writerSettings;
        private string contentType;

        private string _MediaType;
        private MessageVersion _MessageVersion;

        public MustUnderstandOffEncoder(MustUnderstandOffEncoderFactory factory)
        {
            _MediaType = factory.MediaType;
            _MessageVersion = factory.MessageVersion;

            this.writerSettings = new XmlWriterSettings();
            this.writerSettings.Encoding = Encoding.GetEncoding(factory.CharSet);
            //this.writerSettings.ConformanceLevel = ConformanceLevel.Fragment;
            //this.writerSettings.OmitXmlDeclaration = true;
            this.contentType = string.Format("{0}; charset={1}",
                        _MediaType, this.writerSettings.Encoding.HeaderName);
        }

        public override string ContentType
        {
            get
            {
                return this.contentType;
            }
        }

        public override string MediaType
        {
            get
            {
                return _MediaType;
            }
        }

        public override MessageVersion MessageVersion
        {
            get
            {
                return _MessageVersion;
            }
        }

        public override bool IsContentTypeSupported(string contentType)
        {
            if (base.IsContentTypeSupported(contentType))
            {
                return true;
            }
            if (contentType.Length == this.MediaType.Length)
            {
                return contentType.Equals(this.MediaType, StringComparison.OrdinalIgnoreCase);
            }
            else
            {
                if (contentType.StartsWith(this.MediaType, StringComparison.OrdinalIgnoreCase)
                    && (contentType[this.MediaType.Length] == ';'))
                {
                    return true;
                }
            }
            return false;
        }

        public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
        {
            byte[] msgContents = new byte[buffer.Count];
            Array.Copy(buffer.Array, buffer.Offset, msgContents, 0, msgContents.Length);
            bufferManager.ReturnBuffer(buffer.Array);

            MemoryStream stream = new MemoryStream(msgContents);
            return ReadMessage(stream, int.MaxValue);
        }

        public override Message ReadMessage(Stream stream, int maxSizeOfHeaders, string contentType)
        {
            XmlReader reader = XmlReader.Create(stream);
            return Message.CreateMessage(reader, maxSizeOfHeaders, this.MessageVersion);
        }

        public override ArraySegment<byte> WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
        {
            MemoryStream stream = new MemoryStream();
            XmlWriter writer = XmlWriter.Create(stream, this.writerSettings);
            message.WriteMessage(writer);
            writer.Close();

            string decoded = Encoding.UTF8.GetString(stream.ToArray());
            decoded = decoded.Replace("mustUnderstand=\"1\"", "mustUnderstand=\"0\"");
            stream.Close();
            byte[] messageBytes = System.Text.Encoding.UTF8.GetBytes(decoded);
            int messageLength = messageBytes.Length;

            //byte[] messageBytes = stream.GetBuffer();
            //int messageLength = (int)stream.Position;
            //stream.Close();

            int totalLength = messageLength + messageOffset;
            byte[] totalBytes = bufferManager.TakeBuffer(totalLength);
            Array.Copy(messageBytes, 0, totalBytes, messageOffset, messageLength);

            ArraySegment<byte> byteArray = new ArraySegment<byte>(totalBytes, messageOffset, messageLength);
            return byteArray;
        }

        public override void WriteMessage(Message message, Stream stream)
        {
            XmlWriter writer = XmlWriter.Create(stream, this.writerSettings);
            message.WriteMessage(writer);
            writer.Close();
        }
    }

}

While that is all the code I ended up using.  There is one more class I wanted to share for completeness.  During my research, at one point I thought my problem was stemming from passing in username and password security credentials in the header without a nonce and time stamp.

This is a feature Microsoft apparently purposefully excluded because they felt it was less secure and should not be promoted.  Fortunately I ended up not needing it, but I feel for those who do need it and must deal with Microsoft taking the ability away from them because they think they know better than the developer.

Here is the class I ended up using successfully to fix this particular issue.  I copied it almost verbatim from Rick Strahl's Blog.

Custom Credentials Classes
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.ServiceModel.Security;
using System.IdentityModel.Tokens;
using System.Security.Cryptography;

namespace Services
{
    public class WsNonceCustomCredentials : ClientCredentials
    {
        public WsNonceCustomCredentials() { }

        protected WsNonceCustomCredentials(WsNonceCustomCredentials cc) : base(cc) { }

        public override System.IdentityModel.Selectors.SecurityTokenManager CreateSecurityTokenManager()
        {
            return new WsNonceCustomSecurityTokenManager(this);
        }

        protected override ClientCredentials CloneCore()
        {
            return new WsNonceCustomCredentials(this);
        }
    }

    public class WsNonceCustomSecurityTokenManager : ClientCredentialsSecurityTokenManager
    {
        public WsNonceCustomSecurityTokenManager(WsNonceCustomCredentials cred) : base(cred) { }

        public override System.IdentityModel.Selectors.SecurityTokenSerializer CreateSecurityTokenSerializer(System.IdentityModel.Selectors.SecurityTokenVersion version)
        {
            return new WsNonceCustomTokenSerializer(System.ServiceModel.Security.SecurityVersion.WSSecurity11);
        }
    }

    public class WsNonceCustomTokenSerializer : WSSecurityTokenSerializer
    {
        public WsNonceCustomTokenSerializer(SecurityVersion sv) : base(sv) { }

        protected override void WriteTokenCore(System.Xml.XmlWriter writer,
                                                System.IdentityModel.Tokens.SecurityToken token)
        {
            UserNameSecurityToken userToken = token as UserNameSecurityToken;
            if (userToken == null) return;

            string tokennamespace = "o";

            DateTime created = DateTime.Now;
            string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ");

            // unique Nonce value - encode with SHA-1 for 'randomness'
            // in theory the nonce could just be the GUID by itself
            string phrase = Guid.NewGuid().ToString();
            var nonce = GetSHA1String(phrase);

            // in this case password is plain text
            // for digest mode password needs to be encoded as:
            // PasswordAsDigest = Base64(SHA-1(Nonce + Created + Password))
            // and profile needs to change to
            //string password = GetSHA1String(nonce + createdStr + userToken.Password);

            string password = userToken.Password;

            writer.WriteRaw(string.Format(
            "<{0}:UsernameToken u:Id=\"" + token.Id +
            "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" +
            "<{0}:Username>" + userToken.UserName + "</{0}:Username>" +
            "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\">" +
            password + "</{0}:Password>" +
            "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" +
            nonce + "</{0}:Nonce>" +
            "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace));
        }

        protected string GetSHA1String(string phrase)
        {
            SHA1CryptoServiceProvider sha1Hasher = new SHA1CryptoServiceProvider();
            byte[] hashedDataBytes = sha1Hasher.ComputeHash(Encoding.UTF8.GetBytes(phrase));
            return Convert.ToBase64String(hashedDataBytes);
        }

    }
}

My final assessment of the situation is that Microsoft really screwed things up by removing this feature from WCF.  However, with a lot of work, it is possible to hack the change back in.  Hopefully Microsoft will fix this problem some day.  Whether it is an oversight or a feature on their part, the decision they made was a bad one for those of us actually having to do the work to undo what they did.

Thursday, October 12, 2017

NetTeirs and Reflection

By now NetTiers is a rather aging templating system.  Many people have complained that it is not being actively maintained and that there are a lot of errors in it, and to some degree they are correct.

NetTiers gets random updates from those using it, but not very frequently.  The code is definitely not bug free, however it is not chalk full of bugs either.  It does rely on Microsoft's EnterpriseLibrary sweet, which is no longer maintained; although the documentation claims this is not a requirement.

But NetTiers has several key features and functionality that had me pick it up after testing multiple data access layer solutions.

  • It generates a direct copy of your database into C# code with just the click of a button.  Very few other data access layers can claim that, and their solutions do not feel as smooth as the Codesmith platform that NetTiers runs on.
  •  During the generation process NetTiers standardizes the table and column names found in your database, so they all start capitalized, words it identifies are capitalized, and table prefixes can be optionally stripped off.
  • It uses meta data from sql regarding indexes and foreign keys to auto generate methods to get data out of the database.
  • That meta data also allows links to be made between objects in the generated c# code so you can load and reference foreign key relationships easily.
  • Stored Procedures can be auto generated for those who want the added security and to avoid dynamic sql.
  • It has a reasonably powerful parameterized sql engine when required.
  • A caching layer allows the developer to cash most result sets that come back from the database without additional work.  Which can be useful for small static tables.
  • It picks up custom stored procedures and adds methods onto the generated objects to allow the developer to call them without any additional effort.
  • There are quite a few template options to allow the developer to customize the resulting code to their environment.
Many data access layers claim features similar to these, although oftentimes with less control over them.  However, there are three features that make NetTiers unique and keep it valuable and relevant:

The direct copy of the database into standard c# code takes a huge load off of the developer who no longer has to write all that plumbing code.  The generated code is in the form of normal C# objects that match the names in your database.  This means there is no guesswork as to what an object name is going to be.  It also means full object oriented and intellisense capable access to every object in the database.

Generated code is compile time safe; so most errors in the database will show up right away and not allow the code to compile.  This code has been used by many people lots of times, so most major bugs have been worked out already.  Having it all generated the same way every time nearly eliminates the ability for a random error to sneak into the data access layer.

One of my favorite features is actually a byproduct of having the data access layer so standardized.  The ability to use reflection heavily.  Because there are so many standards built into nettiers, you can always find the correct get or save methods or primary keys regardless of the object you are working with.  The generated code in these areas all follows the same pattern.

While not perfect ( I have submitted a few corrections back to the code base myself ), the ability to auto generate an entire DAL without effort, and the ability to rely heavily on reflection when using these objects, has made me a believer in NetTiers for many of the projects I work on.

It is not the only solution available.  But for server based C# projects it definitely does shine as a way to drastically reduce development time.



Tuesday, May 16, 2017

WanCam wireless camera HW0036

I made a mistake in getting this camera.  Despite it having the features it advertised, this camera is missing an RJ45 jack.  For security reasons I never buy a camera without a wired connection.



This camera does work correctly, however they made a couple of annoying design decisions when making this model.  First, the http port by default is 81 rather than the standard 80.  Second they also deviated from the standard OnVif ports and choose to use 10080 for this service; and it is not configurable.

It does not support https, which I consider a negative, although not horrible as I rarely use the web interface once configured.

Fortunately you can change the http port, and you can change the default password.  The camera works very well with Milestone.  As long as you only want wireless then this is a good camera, however because of that reason I would never recommend, or purchase another one.

EDIT: Within a week of owning this camera it began to have issues turning on.  A few days later the problem was so bad the camera no longer functioned.  It is possible that I simply got a lemon, or this particular model has a bad design flaw.  I was able to get a partial refund, threw the camera away, and will never get another of this model.

Monday, May 15, 2017

KanKun Wireless Plug

I recently purchased  a wireless KanKun plug, specifically because I heard that it was running a version of OpenWRT and could easily be modified.

I found the primary directions here.

There were a couple of challenges.  The first was getting the plug on the wireless network.  The Chinese directions and app did not function at all, and I tried them multiple times.

The good news is that the plug successfully resets all failed attempts very well with its reset process.  Eventually I was successful following these steps:

- Use a laptop to connect to the wireless network the plug defaults to creating.
- Lookup the Plugs IP based on the DHCP address assigned to the laptop because it did not create the network that most examples were claiming it would.
- Use Putty to SSH into the plug with a username of 'root'.  My default password was p9z34c, however others have claimed it might be admin or 1234.
- Edit the following file with 'vi': /etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '9'
        option hwmode '11ng'
        option path 'platform/ar933x_wmac'
        list ht_capab 'SHORT-GI-20'
        list ht_capab 'SHORT-GI-40'
        list ht_capab 'RX-STBC1'
        list ht_capab 'DSSS_CCK-40'
        option htmode 'HT20'
        option disabled '0'
        option country 'CN'

config wifi-iface
        option device 'radio0'
        option network 'wwan'
        option ssid 'MyWirelessSID'
        option encryption 'psk2'
        option mode 'sta'
        option key 'MyWirelessPassword'

I made sure the channel was set correctly since I run my wireless network on 9 rather than the standard 11.  I also struggled a bit with the encryption since I run WPA2 Personal; I finally got the very logical instructions to set encryption to 'psk2'.

- The second critical file to edit is: /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd59:45ed:8ead:0000:/48'

config interface 'lan'
        option ifname 'eth0'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.145.253'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option proto 'dhcp'
        option ifname 'eth1'

config interface 'wwan'
        option proto 'static'
        option ipaddr '192.168.1.32'
        option gateway '192.168.1.1'
        option netmask '255.255.255.0'

I modified the MAC address in this file, adding the '0000', however leaving it empty would probably have been fine as well from what others have said; and considering it was working prior.  Adding the last 'config interface wwan' section is the most critical.  It is used by the wireless file for interface settings and specifies the static IP network necessary to communicate on this network.

- 'reboot' the plug
- With the plug now successfully connecting to my network I was able to return to my main computer for further configuration.
- My next step was to copy in a simple CGI script someone created to allow me to remotely control the plug.  I did have to modify the following script slightly because my RELAY_CTRL was different from the ones they were using.
- I had to create the folder /www/cgi-bin
- And create the following file: /www/cgi-bin/relay.cgi

#!/bin/sh
echo "Content-Type: text/plain"
echo "Cache-Control: no-cache, must-revalidate"
echo "Expires: Sat, 26 Jul 1997 05:00:00 GMT"
echo

RELAY_CTRL=/sys/class/leds/i-konke\:red\:relay/brightness

case "$QUERY_STRING" in
state) 
case "`cat $RELAY_CTRL`" in
0) echo "OFF"
;;
1) echo "ON"
;;
esac
;;
on) 
echo 1 > $RELAY_CTRL
echo OK
;;
off) 
echo 0 > $RELAY_CTRL
echo OK
;;
esac

- Unfortunately I discovered that the uhttpd service did not exist on my plug.  I had to turn to a very helpful Internet community who was able to give me the required files.  You can download them here.
- Inside the rar file you should see a kkplug folder with the most likely folder structure and the files that go in each folder.  Use WinSCP to copy the files to the plug.
- Run the following command
/etc/init.d/uhttpd enable
- This creates a sym link that causes the plug to execute the startup file on boot.
- 'reboot' again to test and verify that the service will start correctly.

- When the plug comes up you should be able to use a web browser and control the plug in this way.
http://192.168.1.32/cgi-bin/relay.cgi?on