Knewmart Indoor IP Camera

I am always on the lookout for inexpensive yet quality cameras to integrate with my security system.  If you buy a camera today from almost any store it is nearly a given that it will come with an app that you can use to view the camera remotely.  But aside from a few small motion detection perks that is usually about the end of it.  Many cameras do not play well with other cameras, or make it easy and cheap to actually record like a real security system.

For this reason I have come up with the following rules when buying cameras:
1. It must support OnVif.  This is a standards protocol that guarantees the camera will at least try and play nice with most of the security systems out there.
2. It must support H.264 encoding.  This is a compression algorithm that significantly reduces the amount of storage required when recording for long periods.
3. It must have an Ethernet jack.  Many people like the idea of going wireless for convenience, but even wireless cameras need a power cord.  However, wireless has the major downside of being insecure.
  - Not only is it much easier to hack into a wireless feed.
  - But if you put the camera next to your microwave and turn it on you will realize it is a piece of cake to simply overwhelm the camera with radiation, disabling the feed.
  - Also a nice quality camera will be sending so much data over the wireless network it can overwhelm a normal wireless router, effectively rendering your wireless network useless.
  - And even if you decided to give it its own wireless network, you are still cluttering the radio waves which could end up causing various problems for you in the future.
4. It really should allow for a Static IP to be set, it is just easier to manage networks that way.
5. If it is an outdoor camera it must support POE.  Power Over Ethernet allows me to run just one low voltage cable to the camera.  Trying to mess with high voltage extension cords and a power cable just is not worth the trouble.

As a Note.  For outdoor cheap cameras I have had better luck with the few large LED configuration over the numerous small LEDs for night vision.  Cheaper cameras are not made with the same care, and you can end up with the small LEDs not being positioned correctly and washing out your image at night.

Camera technology is moving fast, but after testing quite a few brands I had settled on a really nice outdoor camera that met all my requirements and worked very well indoors also.  But then two new brands came on the scene, Wancam and Knewmart.

Today I am looking at the Knewmart (no model).

Positives
- It fits all my requirements for an indoor camera (eg, it supports OnVif and H.264 and allows for a static IP to be set, and has an Ethernet jack)
- It has wireless built in in case I do ever need it, and the ability to disable it for security if I don't use it.
- For those of us who do not like installing more apps on our phone it has a built in web server so I can configure it from my computer, which I love.  It is the first web interface I have seen on a less than $50 camera that actually works well.
- The web configuration interface is very simple and easy to use and really started me falling in love with this camera immediately.
- The camera ships with DHCP enabled by default, so it auto connects to your wired network when you plug it in.  This is a much better design in my opinion than cameras that try and stand up their own wireless network for configuration.
- It has a default username/password of admin/admin.  It does have a user and guest account, which is nice, but it has no way to disable them, so make sure you change all the passwords.
- For those who are paranoid about configuring their own cameras, it does come with a QR code to auto configure basic connectivity in their native app.
- It has the ability for dual streams at different quality levels, which was unexpected although it is probably becoming more standard in the industry.
- For those who do not run separate NVR software, it has the built in ability to trigger alarms during a time span you schedule and send you an email with a picture of what caused the alarm. However, I did not test this.
- It has little motors so you can pan and tilt it, which is new for the less expensive cameras, and I absolutely love it.  This Knewmart has the widest range of motion I have experienced yet, it Pan: 355°, Tilt: 90°
- It does work with the E-View7 app, which I liked because I already use that app for other cameras.  They also made a special P2PIPC app according to the directions that came with it.
- For those who are not quite ready to built an NVR system, but still want to record some video locally, it does come with a MicroSD card slot.  While this does seem to be a popular feature, it is a feature I have never personally found a good use for.
- It also comes with the ability to talk out of it and hear through it, another set of features I absolutely love in this camera.
- There is port labeled as a headphone jack, although I think the picture is misleading, I believe it is a Microphone jack in case you want better audio reception.  The OnVif api supports this conclusion by listing two microphones.
- I use Milestone's XProtect software, and after enabling OnVif it had no issue connecting to the camera on the first try. Microphone and pan/tilt work great.
- As far as cameras go, this thing actually looks very nice.  I question whether it is a bit too big and bulky for an indoor camera that you typically want to be unobtrusive; but the smooth fluids lines on it help make it a more attractive decoration.  Plus there is the argument that being bigger means better quality.
- It has the built-in ability to contact time.windows.com to set it's own time and handle daylight savings time; you just have to choose the right time zone.  This is a very handy feature that is often overlooked.
- It has the standard Flip and Mirror options for the picture, which are really nice when you want to mount the camera in an odd place, like the ceiling.
- I will give this company credit for effort in support also, this is a nice in-expensive camera, and they respond to support requests.  It may not be American level support, but they also do not leave you hanging which is really nice.

- It does have a reset button, I mention this obvious feature only because I have actually seen cameras without one.
- It seamlessly switches between wireless and wired without requiring a hard reboot.

- And finally it takes a unique approach to wall and ceiling mounting.  Most cameras like this have holes in their bottom you just slide over screws.  This camera ships with a pretty configurable mounting arm.  While I am not sure the aesthetics of it are great, it certainly does allow for extreme flexibility in how and where you mount the camera.

Negatives
- The directions that came with the camera are very basic and intended for a user who only cares about getting the camera running.  If you want to do anything advanced with it you will probably need to use the web interface on your computer and do a little techie research if you do not already understand some of the terminology.
- It does have a small glitch/feature where you have to manually unplug it after making some change to the OnVif settings to actually make OnVif work, it comes enabled by default it just does not work by default.  I have experienced this with the wancam as well.

- I was disappointed that the speaker does not work over OnVif with my XProtect NVR system, but I expected that from other similar cameras I have had.  Hopefully some day they add that feature.
- It does not support POE.  But for an indoor camera that is not a deal breaker for me.

- Like other cameras of this type, when you first turn it on it tests its range of motion, likely to configure the limits in the software.  It has always concerned me that the camera trying to go past its limit during these tests will eventually burn out the motor, but so far that has not happened.
- It does not support Https, however for those who are that paranoid about security (like me) they are probably running their cameras on a closed internal secure network anyway, so it would not be much of an issue.
- If you tilt the camera too far down then its night vision LEDs reflect on itself causing the image to blur white.

- The default IRCut value seems to be too low, in low light the camera starts flickering the IR on and off.  You can either fiddle with the settings, or just switch it to manual mode.
- It is not very easy to unplug a network cable once plugged into it, especially with bigger fingers.
- When used via OnVif with the XProtect software, there is an error in the OnVif protocol when used over wireless that causes a lag in the image frames triggering XProtect to constantly report it as disconnected.  Oddly enough, you can get around this error by plugging it into a network cable, waiting for the transition, then unplugging the network cable.  It will then work correctly for awhile.

All in all, this is a great camera that can support a range of users from those starting out, to those who are quite a ways along in building a system.  If it were not for the larger size (roughly double it's WanCam counterpart) it would become my go-to indoor camera for home systems.  However, it beats out other inexpensive cameras in so many areas that it is still going to be at the top of my list for consideration when I recommend cameras to people

NetTiers Ambiguous match found

I recently ran into a bug that took me longer than it should have to figure out.

I was migrating an old code base to CodeSmith's NetTiers and started getting the error "Ambiguous match found".  Results from the Internet all pointed the finger at Microsoft's .net, albeit earlier versions than version 4.5 that I was using.

While I did become convinced that it was a Microsoft issue relating to two object attributes with the same name just different letter casing, that did not solve my issue for me because my code was all being auto generated by NetTiers.  NetTiers was using reflection to get some attribute values, so I figured one of my table or column names was conflicting with some internal variable that NetTiers was using.

The solution turned out to be rather simple.  I ran the code in debug mode, triggered the crash, and started inspecting the variables in play.  Particularly the one NetTiers was trying to find using reflection.  It turned out that NetTiers was looking for an attribute named "Item", which corresponded to a table column of the same name, and was blowing up in the attempt.  I renamed that column in my database, regenerated the code, and the problem was solved.

Downloading Ancestry.com Media

Ancestry.com has never created a good system to back up the data that is on their site. The only option people have is to download the GEDCOM file, which is often a mere fraction of the data they have invested in the site.

For years now my family has been investing time in building a well documented family tree on ancestry.com, and I have made sure to frequently backup the GEDCOM file so we didn't lose the data.  With thousands of media files on the site I have become increasingly concerned about the loss of a major portion of our research if I was unable to download them.

To combat this problem I finally wrote a small Chrome extension that allows you to screen scrape your media files in an automated fashion.  It is not perfect, but considering it is the only way to get your data, it is nice to have.

The extension downloads the files in mass and dumps them all into your default downloads folder.  So, you should clear all files out of your default downloads folder before using this extension so you know that all downloaded files belong to Ancestry.

After installing this extension, you need to:

- log into your ancestry.com account

- open up your tree

- open up the Media page, which is one of the items on the menu inside the tree

- click on either the Photos or Stories tab, currently the All, Audio, and Video tabs are not supported

- OPTIONAL: click on the page you want to resume downloading at.  This is useful if you need to stop downloading for a period of time, say if you only want to run the downloads during the night.  You will likely end up with a few duplicate files if you attempt this.

- click on the Extension icon in your Chrome browser upper right corner, and click the Download Media button

If your browser is asking you to save every file then the extension will not be able to correctly calculated the required download time and it will go on to the next page before you are done. It is expecting the download to automatically start.  To fix this problem, go into your browser settings and adjust it so downloads do NOT always prompt you for a save location. Instead set a default location for it to use every time. The only way I could fix this issue would be to create a setting that allows the user to manually click the next page button instead of having it automated. And I think that would kind of defeat the point of the extension.

If the downloads are occurring too quickly and swamping your computer then let me know.  I am planning on allowing the download speed to be adjusted in the future.  Currently a file is downloaded every 2 seconds, which works well for me, but for people who upload lots of large files it will not work as well for.

NOTE: There is a small chance that using this is against Ancestry's terms of use.  Their terms of use forbid scraping tools, however the language seems directed at automated scripts that run in the background.  This tool is simply a browsing aid performing clicks for you, but your browser is still open and doing all the browsing and preventing you from doing other browsing simultaneously.

EDIT 10/2018:

Just released a new version that works with Ancestry's latest photo hosting method.  In order to get it working again the extension is going to need permissions to open and close windows as well as download files.  When you first try and use the photo downloading feature Chrome will likely popup a few requests asking you to let the extension do downloads and open and close windows.  So expect that as you try to get it going.

This version also handles the Start/Stop downloading sensing better.  Although, I have noticed that sometimes it takes one or two tries to get it to stop downloading depending on at what point in its execution you click the stop button.

LastPass Password Manager

Most people I have met know that long complex passwords are a good idea.  But very few people know of a good way to keep track of all their passwords.  I personally have well over 200 accounts that I need to track passwords for.  Many people, myself included, get stuck in the habit of picking out a handful of passwords and using them for everything.

Over the years I have tried a few password managers including KeyPass and PassPack.  They all had nice features, but they all ended up being such a pain that I was never able to use them to create truly random unique passwords for every account I have.  KeyPass did not sync across computers, PassPack had a very limited free version, and the issues go on.

Then I discovered LastPass.  It took me a month of usage to fully start trusting it, but slowly I have started using more and more features of it.  The best part is the amount of features they offer for free.  And their first paid level ( that many people will want ) is only $12/yr at the moment.  So price for features was exactly what I was looking for.

LastPass syncs across all my computers, or phones for free (paid version can do both at the same time).  It auto fills my website passwords with a Chrome plugin.  It has been gamified to show me my overall security score and compare it to the score of other users.  It supports lots of dual factor authentication options.  It allows me to re-prompt for my password and do other security enhancements on a per password basis.  It even has built in form fills, both generic and tailored to sites that require more than just a username and password to login.  And critically, it allows me to backup all the data in my account.

Having an easy backup system is critical for me, because no matter how good software is, and how much they claim they are not going anywhere, anything can happen.  Their website could crash, they could go out of business, my encrypted data could get corrupted, I could forget my master password, etc.  So I use the backup feature frequently to export a plain text copy of all my data including passwords, I then encrypt the file and store it.  So if anything every happens, or a better software comes along, I can easily recover using my backup file.

In addition to a good backup system, I also wanted to be sure they could not be hacked.  Unfortunately they have been hacked at least twice.  However, the articles and responses to these hacks were so good that it made me feel even more secure using the software; no critical data was lost, users could easily remove the danger by changing the one piece of lost data, and that lost data was encrypted so heavily that it was unlikely anyone would figure it out anyway.  I did use their advanced settings to increase my security above the default level though.

A few things I did to make my account more secure:
- Make sure all my client browser plugins log me off after a reasonable period of inactivity
- Go into my vaults advanced settings:
- - increase the client-side rounds to something greater than 50k. Due to LastPass's speed warnings I did this in increments testing the speed both on my phone and computer.  I also backed up my data first in case the re-encryption corrupted everything.
- - restricted logins to just my country
- - added dual factor authentication
- - reduced my website logoff settings down to a day

Studying for Microsoft 70-463 ( Implementing a Data Warehouse with Microsoft SQL Server 2012 )

For those of you out there who wish to get your MCSA SQL Server 2012, this is my path of choice, I hope it helps some of you out:

1. Take the free Jump Start course from MVA
2. Practice and brush up on features using the Microsoft Virtual Lab for Exploring MDS and DQS, about half way down the page.
3. Pull together 60 practice questions. which you can take as well.
4. Pull together 60 more practice questions.
5. Pull together 60 more practice questions.
6. Pull together a few more practice questions.
7. Found someone else's 86 practice questions.
8. Buy the MeasureUp practice exam and hammer it to death.
9. Go take the test.

Microsoft even offers their own practice test for this exam if you wish to purchase it.  The real exam may or may not include any of the questions compiled in any set of practice questions.

That said, practice tests are only a means of validating a persons knowledge.  The MVA class and Virtual Lab are where the actual learning should take place.

Cloning / Hacking RF remote controls using Arduino

I've been on a smart house kick lately trying to add every sensor I can find to my house that falls within my budget.  Due to budgeting issues I am always looking for new ways to do things cheaper.

Basic sirens coupled with Normal Open and Normal Closed sensors were the obvious first step because they are cheap and fairly easy to understand and implement.  And they give you rudimentary detection and feedback capabilities.  But I want more, and to get more I have been exploring all the IP based options available for controlling lights, outlets, and other household devices.

Unfortunately, in part to the current smart house craze, any device that supports traditional smart house technology like z-wave, wifi, and zigbee are extremely expensive.  But then I thought about traditional remote controls.

The vast majority of remotely controlled devices use standard Radio Frequencies (RF), usually in the 315 or 433MHz range.  Most people do not consider this to be smart house related because they only work up to a range of a few hundred feet, are usually unidirectional in communication, have no interface to be controlled through the Internet, and absolutely zero security around the protocol.

There isn't much I can do about the lack of security, that's just a fact of life about cheap remote controls.  However, I did not see any reason why I could not make a computer connected radio that sent out the same wave patterns as my remotes.  With these cloned signals I would then be able to remotely and centrally control anything with an RF receiver in it.

After days of reading about the subject and other peoples attempts to do the same thing, I bought an arduino and a 315MHz receiver/transmitter set.  Three days later after a lot of signal analysis, trial and error, I finally got my first signal cloned.  I now have the ability to repeat the process for any RF remote that operates on the 315MHz frequency with a fixed code and control its item remotely as a fully integral part of my smart house.

The process involved:
- writing a program for arduino that sampled one of the analog pins on a tight loop tracking the strength and duration of the signal fluxuations.
- I then moved this data into excel where I normalized it to determine the binary code it was sending.
- Converted that binary code into an integer mapping table for condensed storage.
- Wrote a program to use a digital out pin to send that signal to the transmitter which turned it back into an analog wave form in the air.

To be fair, both of the base programs I started with someone else who knew the arduino far better wrote, I just modified them to be more efficient for my use case.

Here is the code for the RF Receiver to capture signals:

 #define analogPin A0 // RF data pin = Analog pin 0  
 byte dataBuffer[512];  
 int dataCounter = 0;  
 int maxSignalLength = 255;  
 const unsigned int upperThreshold = 100;  
 const unsigned int lowerThreshold = 80;  
 void setup() {  
  Serial.begin(115200);  
  // wait until a LOW signal is received  
  while(analogRead(analogPin) < 1) { }  
  // got HIGH; read the rest of the data into dataBuffer  
  for (int i = 0; i < sizeof(dataBuffer); i = i+2) {  
   // LOW signals  
   dataCounter = 0;  
   while (analogRead(analogPin) > upperThreshold && dataCounter < maxSignalLength)  
    dataCounter++;  
   dataBuffer[i] = dataCounter;  
   // HIGH signal  
   dataCounter = 0;  
   while(analogRead(analogPin) < lowerThreshold && dataCounter < maxSignalLength)  
    dataCounter++;  
   dataBuffer[i+1] = dataCounter;  
  }  
  Serial.println("LOW,HIGH");  
  delay(20);  
  for (int i = 0; i < sizeof(dataBuffer); i = i+2) {  
   Serial.print(dataBuffer[i]);  
   Serial.print(",");  
   Serial.println(dataBuffer[i+1]);  
   delay(20);  
  }  
 }  
 void loop() { }  

Receiving the signals is pretty straight forward.  You have to wire your receiver into the arduino.  If you are holding it with the top of the board facing you, then the pins are from left to right: Power, Data, Data, Ground.  Hook the Power (or Vcc) into a 5V pin, 3.3V should work as well.  This code is supposed to be used with the second Data pin, but it doesn't really matter, the HIGH/LOW values will just be reversed if you use the other pin.

Upload the code to the arduino, it will automatically run since it is all in the setup function.  Press Ctrl+Shift+M to bring up the serial monitor, and make sure you set it to receive at 115200.

Immediately press the button on your remote.  Once the data appearing on the screen it is too late, the capture has already taken place.  I had to do this process several times, playing with the data each time until I finally got a capture that was clean enough I was able to find a good pattern in it.

Next you need to paste the data into excel, or some other text editor and start manipulating it.  You need to split it into two columns, a HIGH and  LOW column, then count the number of occurrences of every value one column at a time.  I wrote a macro to do this for me (the guy who's tutorial I followed wrote a php script for himself).

Next sort the value low to high based on the value that was being counted.

This is the really tricky part now.  You need to identify the signal groupings.  Most basic remotes are only going to have two.  A common short burst and a common long burst; or smaller number and larger number.  So pick the low value and the high value that occur most frequently and group the other more random values in on those two.  I picked the values manually, but had Excel re-write the originally values into two new "normalized" columns for me.

Then you re-combine the two new "normalized" columns back into a single column, perform a Unique function on them, and number each unique value.  The purpose for this is we need an easy way to identify the values, copying and pasting all those values pairs into code over and over would just get too difficult very fast.

Finally you make another column which pairs the two "normalized" columns up with their newly assigned unique value.  Combine all those values into a comma delimited string.  And start searching for patterns.  I often find that the unique value column is a better place for me to go pattern hunting in.  Values of 0 and 255 are timeouts and can usually be ignored; if they happen in a very specific pattern inside of the pattern you end up identifying then it could indicate a missing value that you need or could try guessing at.

Here is what my excel sheet looked like through one of these processes:

And here is the Code to re-transmit the signal, however it will have to be modified for your values:

 #define transmitPin 7 // RF transmit pin = Digital pin 7  
 int ding[] = {1,2,3,2,3,2,3,2,2,2,2,2,3,2,3,2,3,2,2,3,3,2,2,2,2};  
 // Play with this value  
 int timeDelay = 155;  
 int signalPin = 13;  
 int numberOfTransmissions = 19;  
 int numberOfSignalInTransmission = (sizeof(ding)/sizeof(int)) - 1;  
 void setup() {  
   Serial.begin(9600);  
   pinMode(transmitPin, OUTPUT);  
   pinMode(signalPin, OUTPUT);  
   delay(2000);  
  // wait 3 seconds, then transmit the payload  
  int highLength = 0;  
  int lowLength = 0;  
  for(int y = 105; y < 415; y=y+10) {  
  for(int x = 0; x < numberOfTransmissions; x++) {  
   if(x % 2 == 0)  
    digitalWrite(signalPin, HIGH);   
   else  
    digitalWrite(signalPin, LOW);  
   for(int i = 0; i < numberOfSignalInTransmission; i++) {  
    switch(ding[i]) {  
     case 1:  
      lowLength = 3;  
      highLength = 119;  
     break;  
     case 3:  
      lowLength = 3;  
      highLength = 11;  
     break;  
     case 2:  
      lowLength = 11;  
      highLength = 3;  
     break;  
     case 4:  
      lowLength = 5;  
      highLength = 5;  
     break;  
     case 5:  
      lowLength = 42;  
      highLength = 2;  
     break;  
    }  
    // transmit HIGH signal  
    digitalWrite(transmitPin, HIGH);    
    delayMicroseconds(highLength*y);  
    // transmit LOW signal  
    digitalWrite(transmitPin,LOW);    
    delayMicroseconds(lowLength*y);   
   }  
  }  
  }  
 }  
 void loop() { }  

The transmitter comes with 3 or 4 pins depending on if it has an onboard antenna or not.  The antenna is absolutely critical; I had a receiver that came without one and it would rarely pick up any signals unless I held a bit of mettle over the solder point where the antenna was supposed to be.

The pins from left to right if you are holding it facing you are: Digital receiver (hook to digital pin 7 for the code above), Power (Vcc 3.3V or 5V, they can usually go up to 12V for increased range), Ground.

The code above is designed with three nested for loops.

The inner most simply takes the ding[] array, converts it back into the values from the normalized columns in your excel document using a case statement, and transmits them. I did not need the last two case statements in this instance.

The next one out specifies how many times you want to send the pattern; most remotes transmit a repeating pattern several times so there is no point is storing multiple copies of the same pattern if we can just loop through it several times.

The outermost loop is very important.  It adjusts the length of the signals you are sending out, how long each high/low signal should be held for before moving on to the next.  The remotes I have seen use a value in the high 200s or low 300s for this.  Finding the precise number can be quite difficult though, so looping through a range that increments by 10 microseconds is a much faster way to at least figure out if you are on the right track before trying to narrow the signal down.

------------------------
Unfortunately it looks like the website I used to figure this all out has gone offline, which is a bummer because it had very detailed step-by-step instructions and was the only set of instructions I could find that I was able to understand clearly enough to figure this all out.

I believe this is the original site I got my tutorial from that is no longer active.
However, here is another site that looks almost as good.

3D printer - Prusa i3 - Repetier - Basic Configuration

When I first built my Prusa i3 I got lucky enough to find someone who had uploaded a pre-configured copy of the repetier firmware.  I uploaded his firmware to my printer and it just worked, like magic.

Unfortunately I ran into a hickup when helping my son with his arduino based RC car.  I accidentally uploaded his arduino program to my printer.  And when I looked for the pre-configured firmware again I was unable to find it.

So, I downloaded the latest and greatest repetier firmware with all default parameters, uploaded it, and kept my fingers crossed.  After about a week of experimentation I realized that my end stops were the only thing not functioning correctly, they seemed to be placed on the wrong side of the axis for some reason.

Another week of playing with settings and no success I finally ran across this post.  All this time I had been focused on my Endstop configuration, it had never occurred to me that was not where the problem lay.

I jumped down to the XYZ movements area, located :
#define INVERT_X_DIR 0
#define INVERT_Y_DIR 0
#define INVERT_Z_DIR 0

and changed them to:
#define INVERT_X_DIR 1
#define INVERT_Y_DIR 1
#define INVERT_Z_DIR 1

And I was back in business, everything was working they way it was before.  This time I backed up my configuration in case this happens again.