Thursday, May 5, 2016

Studying for Microsoft 70-463 ( Implementing a Data Warehouse with Microsoft SQL Server 2012 )

For those of you out there who wish to get your MCSA SQL Server 2012, this is my path of choice, I hope it helps some of you out:

  1. Take the free Jump Start course from MVA
  2. Practice and brush up on features using the Microsoft Virtual Lab for Exploring MDS and DQS, about half way down the page.
  3. Pull together 60 practice questions. which you can take as well.
  4. Pull together 60 more practice questions.
  5. Pull together some more practice questions.
  6. Buy the MeasureUp practice exam and hammer it to death.
  7. Go take the test.

For those na-sayers out there who believe that practice questions are some form of cheating.  Quizzing or flash cards have been used as a method of making sure one knows a subject for as far back as I can remember.  Microsoft even offers their own practice test for this exam if you wish to purchase it.  The real exam may or may not include any of the questions compiled in any set of practice questions.

That said, practice tests are only a means of validating a persons knowledge.  The MVA class and Virtual Lab are where the actual learning should take place.

Friday, March 11, 2016

Cloning / Hacking RF remote controls using Arduino

I've been on a smart house kick lately trying to add every sensor I can find to my house that falls within my budget.  Due to budgeting issues I am always looking for new ways to do things cheaper.

Basic sirens coupled with Normal Open and Normal Closed sensors were the obvious first step because they are cheap and fairly easy to understand and implement.  And they give you rudimentary detection and feedback capabilities.  But I want more, and to get more I have been exploring all the IP based options available for controlling lights, outlets, and other household devices.

Unfortunately, in part to the current smart house craze, any device that supports traditional smart house technology like z-wave, wifi, and zigbee are extremely expensive.  But then I thought about traditional remote controls.

The vast majority of remotely controlled devices use standard Radio Frequencies (RF), usually in the 315 or 433MHz range.  Most people do not consider this to be smart house related because they only work up to a range of a few hundred feet, are usually unidirectional in communication, have no interface to be controlled through the Internet, and absolutely zero security around the protocol.

There isn't much I can do about the lack of security, that's just a fact of life about cheap remote controls.  However, I did not see any reason why I could not make a computer connected radio that sent out the same wave patterns as my remotes.  With these cloned signals I would then be able to remotely and centrally control anything with an RF receiver in it.

After days of reading about the subject and other peoples attempts to do the same thing, I bought an arduino and a 315MHz receiver/transmitter set.  Three days later after a lot of signal analysis, trial and error, I finally got my first signal cloned.  I now have the ability to repeat the process for any RF remote that operates on the 315MHz frequency with a fixed code and control its item remotely as a fully integral part of my smart house.

The process involved:
- writing a program for arduino that sampled one of the analog pins on a tight loop tracking the strength and duration of the signal fluxuations.
- I then moved this data into excel where I normalized it to determine the binary code it was sending.
- Converted that binary code into an integer mapping table for condensed storage.
- Wrote a program to use a digital out pin to send that signal to the transmitter which turned it back into an analog wave form in the air.

To be fair, both of the base programs I started with someone else who knew the arduino far better wrote, I just modified them to be more efficient for my use case.

Here is the code for the RF Receiver to capture signals:

 #define analogPin A0 // RF data pin = Analog pin 0  
 byte dataBuffer[512];  
 int dataCounter = 0;  
 int maxSignalLength = 255;  
 const unsigned int upperThreshold = 100;  
 const unsigned int lowerThreshold = 80;  
 void setup() {  
  // wait until a LOW signal is received  
  while(analogRead(analogPin) < 1) { }  
  // got HIGH; read the rest of the data into dataBuffer  
  for (int i = 0; i < sizeof(dataBuffer); i = i+2) {  
   // LOW signals  
   dataCounter = 0;  
   while (analogRead(analogPin) > upperThreshold && dataCounter < maxSignalLength)  
   dataBuffer[i] = dataCounter;  
   // HIGH signal  
   dataCounter = 0;  
   while(analogRead(analogPin) < lowerThreshold && dataCounter < maxSignalLength)  
   dataBuffer[i+1] = dataCounter;  
  for (int i = 0; i < sizeof(dataBuffer); i = i+2) {  
 void loop() { }  

Receiving the signals is pretty straight forward.  You have to wire your receiver into the arduino.  If you are holding it with the top of the board facing you, then the pins are from left to right: Power, Data, Data, Ground.  Hook the Power (or Vcc) into a 5V pin, 3.3V should work as well.  This code is supposed to be used with the second Data pin, but it doesn't really matter, the HIGH/LOW values will just be reversed if you use the other pin.

Upload the code to the arduino, it will automatically run since it is all in the setup function.  Press Ctrl+Shift+M to bring up the serial monitor, and make sure you set it to receive at 115200.

Immediately press the button on your remote.  Once the data appearing on the screen it is too late, the capture has already taken place.  I had to do this process several times, playing with the data each time until I finally got a capture that was clean enough I was able to find a good pattern in it.

Next you need to paste the data into excel, or some other text editor and start manipulating it.  You need to split it into two columns, a HIGH and  LOW column, then count the number of occurrences of every value one column at a time.  I wrote a macro to do this for me (the guy who's tutorial I followed wrote a php script for himself).

Next sort the value low to high based on the value that was being counted.

This is the really tricky part now.  You need to identify the signal groupings.  Most basic remotes are only going to have two.  A common short burst and a common long burst; or smaller number and larger number.  So pick the low value and the high value that occur most frequently and group the other more random values in on those two.  I picked the values manually, but had Excel re-write the originally values into two new "normalized" columns for me.

Then you re-combine the two new "normalized" columns back into a single column, perform a Unique function on them, and number each unique value.  The purpose for this is we need an easy way to identify the values, copying and pasting all those values pairs into code over and over would just get too difficult very fast.

Finally you make another column which pairs the two "normalized" columns up with their newly assigned unique value.  Combine all those values into a comma delimited string.  And start searching for patterns.  I often find that the unique value column is a better place for me to go pattern hunting in.  Values of 0 and 255 are timeouts and can usually be ignored; if they happen in a very specific pattern inside of the pattern you end up identifying then it could indicate a missing value that you need or could try guessing at.

Here is what my excel sheet looked like through one of these processes:

And here is the Code to re-transmit the signal, however it will have to be modified for your values:

 #define transmitPin 7 // RF transmit pin = Digital pin 7  
 int ding[] = {1,2,3,2,3,2,3,2,2,2,2,2,3,2,3,2,3,2,2,3,3,2,2,2,2};  
 // Play with this value  
 int timeDelay = 155;  
 int signalPin = 13;  
 int numberOfTransmissions = 19;  
 int numberOfSignalInTransmission = (sizeof(ding)/sizeof(int)) - 1;  
 void setup() {  
   pinMode(transmitPin, OUTPUT);  
   pinMode(signalPin, OUTPUT);  
  // wait 3 seconds, then transmit the payload  
  int highLength = 0;  
  int lowLength = 0;  
  for(int y = 105; y < 415; y=y+10) {  
  for(int x = 0; x < numberOfTransmissions; x++) {  
   if(x % 2 == 0)  
    digitalWrite(signalPin, HIGH);   
    digitalWrite(signalPin, LOW);  
   for(int i = 0; i < numberOfSignalInTransmission; i++) {  
    switch(ding[i]) {  
     case 1:  
      lowLength = 3;  
      highLength = 119;  
     case 3:  
      lowLength = 3;  
      highLength = 11;  
     case 2:  
      lowLength = 11;  
      highLength = 3;  
     case 4:  
      lowLength = 5;  
      highLength = 5;  
     case 5:  
      lowLength = 42;  
      highLength = 2;  
    // transmit HIGH signal  
    digitalWrite(transmitPin, HIGH);    
    // transmit LOW signal  
 void loop() { }  

The transmitter comes with 3 or 4 pins depending on if it has an onboard antenna or not.  The antenna is absolutely critical; I had a receiver that came without one and it would rarely pick up any signals unless I held a bit of mettle over the solder point where the antenna was supposed to be.

The pins from left to right if you are holding it facing you are: Digital receiver (hook to digital pin 7 for the code above), Power (Vcc 3.3V or 5V, they can usually go up to 12V for increased range), Ground.

The code above is designed with three nested for loops.

The inner most simply takes the ding[] array, converts it back into the values from the normalized columns in your excel document using a case statement, and transmits them. I did not need the last two case statements in this instance.

The next one out specifies how many times you want to send the pattern; most remotes transmit a repeating pattern several times so there is no point is storing multiple copies of the same pattern if we can just loop through it several times.

The outermost loop is very important.  It adjusts the length of the signals you are sending out, how long each high/low signal should be held for before moving on to the next.  The remotes I have seen use a value in the high 200s or low 300s for this.  Finding the precise number can be quite difficult though, so looping through a range that increments by 10 microseconds is a much faster way to at least figure out if you are on the right track before trying to narrow the signal down.

Unfortunately it looks like the website I used to figure this all out has gone offline, which is a bummer because it had very detailed step-by-step instructions and was the only set of instructions I could find that I was able to understand clearly enough to figure this all out.

I believe this is the original site I got my tutorial from that is no longer active.
However, here is another site that looks almost as good.

Wednesday, January 13, 2016

3D printer - Prusa i3 - Repetier - Basic Configuration

When I first built my Prusa i3 I got lucky enough to find someone who had uploaded a pre-configured copy of the repetier firmware.  I uploaded his firmware to my printer and it just worked, like magic.

Unfortunately I ran into a hickup when helping my son with his arduino based RC car.  I accidentally uploaded his arduino program to my printer.  And when I looked for the pre-configured firmware again I was unable to find it.

So, I downloaded the latest and greatest repetier firmware with all default parameters, uploaded it, and kept my fingers crossed.  After about a week of experimentation I realized that my end stops were the only thing not functioning correctly, they seemed to be placed on the wrong side of the axis for some reason.

Another week of playing with settings and no success I finally ran across this post.  All this time I had been focused on my Endstop configuration, it had never occurred to me that was not where the problem lay.

I jumped down to the XYZ movements area, located :
#define INVERT_X_DIR 0
#define INVERT_Y_DIR 0
#define INVERT_Z_DIR 0

and changed them to:
#define INVERT_X_DIR 1
#define INVERT_Y_DIR 1
#define INVERT_Z_DIR 1

And I was back in business, everything was working they way it was before.  This time I backed up my configuration in case this happens again.

Monday, December 14, 2015

OpenHab and the Adam 60xx module

Now that I have built a pretty good camera security system using the Milestone XProtect software and various IP cameras, I want to take things to the next level.

I have now successfully integrated an Adam 6052 module (Adam 6050 would work as well) with both the XProtect system and a new OpenHab system using the Modbus plugin for the later.  I have looked over multiple smart house technologies and at the moment the OpenHab software seems to be the most flexible and open for integration with multiple vendors.

With this integration I am now able to track when and for how long my doors are opened for using standard cheap NC wired sensors connected to the Adam module.

Tuesday, August 25, 2015

Open Cart upgrade

During a site upgrade of Open Cart from version 4 to version 6 I ran into some problems.  There were of course the normal issues going between two major versions, but I was finally able to overcome those by upgrading first to version 1.4, then to 2.0.

However, I ran into two unexpected issues after successfully reaching version 6.  The first: I was trying to install the pavilion theme and kept running into odd errors.  I was able to resolve these by upgrading to the latest version of the pavilion theme, apparently pavilion does not have its supported version numbers quite up-to-date yet.

The second issue was an unexplained javascript error "Unexpected end of input OK".  I scoured the web, and tried every known fix out there, but none of them worked.  Finally a post here gave me the needed suggestion to turn on error displaying, which is a setting found in the store settings.  Once that was turned on I got an error written to the screen that made a lot more sense and told me I had forgotten to include a new constant variable in the config file.  I fixed that and everything was working good again.

Note: in the settings table is a new category type column called code I believe.  This new column needs to be updated with the values from the old column, none of the upgrade scripts do this.

Wednesday, June 3, 2015

.NET Rounding Error

A recent project of mine was creating a custom reporting engine in .NET since all of the commercial ones were either too expensive or not flexible enough for the projects requirements.  After creating the engine the natural next step was writing reports to run on it; it was during this process that I found a very interesting error in .net 4.

.NET apparently handles calculations differently depending on whether or not they are implicit or explicit.  For example if you were to run this statement:

If( 3 < (0.6F / 0.2F ))

you would expect .net to do the float calculation, then either cast the integer to a float, or round the float to an integer and perform the calculation.  Apparently it does not as the statement above always evaluates to true, when in reality it is false since the integer 3 is the same value as the float 3.0.  My guess is when performing the division there is some sort of math error in the .net engine which results in something like 3.0000000000000000000001 for 0.6F / 0.2F

The solution to this issue is to explicitly force trimming like this:

Int f = (int)(0.6F / 0.2F)
If( 3 < f )

The above statement casts the float calculation to an integer effectively truncating the extra floating data and allowing a correct comparison.  A better solution would probably be to use Math.Round just in case .net decides to evaluate the calculation to just under 3 rather than just over it.

Friday, March 6, 2015

ELEC new wireless talk camera

ELEC has recently come out with a new wireless camera that allows you to talk out of it.  Considering it is only $40 it is far cheaper than any other competition with that feature so I thought I would try it out.  Unfortunately I was in for a big disappointment.

I was initially pretty disappointed that it lacks an RJ45 port to configure it with, i'm guessing it uses bluetooth or it's own wifi hotspot for initial configuration, either way it requires you to download their app onto your phone for initial configuration.  This is the first camera I have seen that does not have an ethernet port, a big shortcoming in many ways in my opinion.

The web interface for it is viewing only, no configuration changes, which means I need to use my phone for any administrative changes which I do not like.

Their software has a spot for a static IP address, but when I tried to put one in it would not save.  After contacting their support I was told it only supports DHCP.  Another huge shortcoming.  Granted you could put a static reservation in your router for it, so I could live with this if I had to.

Support for elec on the web does not exist yet, so if you get one you will be relying on blog posts like this one, your own whit, or on their support which sometimes responds and sometimes does not.  If they do respond you have to ask only one simple little question at a time, they don't seem to be able to actually read emails.

I had a momentary spark of hope when I attempted adding it to my XProtect system as a generic OnVif device and it was actually recognised.  Unfortunately, XProtect completely loses connection with it if the microphone is enabled (I have covered that issue in my post about dlink products), and still loses connection for a couple of seconds every 30 seconds or so even with the microphone disabled.  I'm not sure who's fault that is just yet. (see below for fix)

Unless I get some more of these issues figured out I will not be buying another Model: EL -Wini001 720P wifi security camera.

Edit: Using wireshark I was able to get the url for the video feed as rtsp://

Using that I setup a Universal driver in XProtect with:
Codec: H264
Streaming Mode: RTP (UDP)
Keep alive: Default
Connection URI: live0.264
RTSP port: 554

So far I have not lost connection to the camera, as long as the microphone is disabled in XProtect.  Wireshark indicates that the audio is G.711 PCMU, however I have not gotten that to work yet.  At least this camera is not a total loss anymore, if I could get the audio to work I think I would even call it a decent camera.  At 1280 x 720 the video quality is definitely far superior to what the dlinks offer.

The connection URI seems to be live0.264 or live1.264 depending on which of the two feeds the camera offers you want to tap into.

It looks like I was able to get the OnVif to work for this camera.  For some reason changing some things in the config resets the cameras protocol from RTP/UDP to RTP/RTSP/HTTP/TCP.  Simply changing that setting back again to RTP/UDP fixes the issue and the camera works great.