Wednesday, November 26, 2008

Asp.net unit tests with nunit - config file

I have been trying to get more into testing lately. I work for a company that isn't really a development shop and I have never been exposed to proper testing methods in real life. So it has been up to me and my teammates to get a test environment setup and working without running up a huge bill.

We started with AutomatedQA's TestComplete product, which is very nice and helps us automate the macro type tasks.

I recently discovered that Visual Studio has builtin support for auto generating a good portion of Unit tests. Unfortunately the MSTest.exe file isn't stand alone. Since we really don't want to buy another license just to run unit tests I decided to check out NUnit which seemed to advertise exactly what we needed.

After a bit of work and learning I got some basic unit tests passing in NUnit. Then I tried to create tests for some of our database method calls. That's when things started blowing up. The error seemed to stem from the methods being unable to find the web.config file.

Reading around a little it looks like since I was running things in NUnit .NET wasn't looking for a config file with the name web.config. After taking half a dozen incorrect guesses based on various blog posts, I grabbed the good old tool File Monitor from sysinternals.com

I used that to figure out that it was looking for a file called unittests.config. I made a copy of my web.config file, renamed it to the desired name, and placed it in the folder where the nunit process was looking.

Now things are working much better, I still have a few bugs to workout, but I am no longer getting the config file error.

As a side note, in the NUnit GUI, under the Project/Edit menu item there is a configuration setting called "Configuration File Name" set to unittests.config. I have a strong hunch that setting controls the name of the file it is looking for. I did try and play with that setting earlier, however nunit kept throwing errors since the file didn't exist so I gave up; might have to look into it again.

Tuesday, November 25, 2008

ClassNotes_2008-11-17



Test Complete – Training Notes
Quick Info

The trainer had trouble deviating from his lecture path so most questions
were put off.
The web wireless login was:
User: AutoQA
Pass: AutoQA
January
15 a new indepth TC book is being released. The instructor promised
me a paperback copy, but didn’t write it down so I doubt he
will remember. I would like a copy of this book when it comes out.
An advanced class is also starting up about this time as well.
Error count comparisons:
- Vista shipped with 73 thousand known bugs.
- the current release of the Mac OS shipped with 25 thousand known
errors.
- NT4 shipped with 120 thousand known errors.
Sqaforums.com is a great reference for the test complete software, probably better
than aqa’s site itself. Sqaforums is monitored by aqa staff.
Testcompletetv is a website that has short video tutorials covering various things
in test complete.

Test Complete
  • It would be a good idea to start off with simple and small tests and
    slowly add features keeping individual tests and functions simple.
  • Absolutely everything you can do in the GUI can be done in code.
  • Smoke Tests are brief tests to make sure the build is not DOA (Dead On
    Arrival).
  • Code Explorer whos stats on code, it also gives a list of functions in
    each unit.
  • Shortcuts:
    -
    Ctrl + Shift + # = bookmark code lines.
    - Ctrl + # = goto
    bookmark.
    - Shift + F2 = kills/stops TC during a run.
    - Ctrl +
    Shift + Space = brings up the parameter completion prompt.
    -
    Click on keyword and press F1 = go to documentation for the
    keyword.
    - Ctrl + J = brings up the available templates for
    insertion
  • Object Explorer:
    - has a blue arrow next to properties if the attribute is read only.
    - right click on ‘sys’ to get “object properties”.
    - target icon means probable open application.
  • Object Browser:
    - the occasional params button allows us to get values for various index’s inside the object.
    - name mapping is reached by right clicking on an object.
  • Process Explorer: right click to filter visible processes.
  • Settings:
    - Log file path, default project properties – all.
    - Tested apps list, allows you to set the number of instances of a tested app
    that are allowed to run before reusing an instance. Default is 1.
    - Docking, the default can be restored if too many windows get messed
    up or closed.
  • Project level variables save their state to an xml file to maintain between
    runs or even system reboots. They are the only communication that
    projects can have between each other in a project suite.
  • Jscript is the most powerful and easy to use language in TC.
  • C# script is a subset of Jscript
  • DelphiScript is what TC is written in and pretty powerful too.
  • VBScript is almost has powerful as Jscript, just a few limitations.
  • Name mapping:
    - a very powerful concept for controls whose nesting
    level/ path changes, but currently a very brittle implementation.
    - to modify mapping layers, make sure the code is using Aliases, then
    delete the named mapping and recreate the correct mapping making
    sure to recreate the same Aliases that were in use before.
  • Wait(ing) for methods wait for an object to terminate.
  • Log:
    - Lock Event, set to true to remove events from the log. If test
    errors out then last 20 or so events are put back into the log.
    - Log.Picture takes a window handle ( not a process handle ) and
    stores a picture in the log. Most of the time it can simply be
    passed an object and it will take a picture of just that object.
    - Log.AppendFolder puts a ‘+’ sign in the Log.
    Log.PopLogFolder bring the logging back up a level outside of a ‘+’
    sign.
  • CPU will skip any waits or events that take less than 16ms to run. This
    is a hardware issue, not a TC feature.
  • Missing Images on a webpage can be found in IE using Image Find to find the
    red X of a missing img. The red X will need to be stored in the
    image stores first. This won’t work with firefox since it
    doesn’t display red X’s. A different image will need to
    be stored for different versions of IE since they don’t likely
    use the same image.
  • Object Compare, compares multiple values and keeps a copy of the object in
    the project stores. Another advantage of this is the error messages
    will tell you what is wrong as well as the incorrect and correct
    values. Only worth using for complex comparisons.
  • Unit Testing:
    - in TC create a new MSTest and load the dll path, leave
    the path to a shared location so it doesn’t need to be
    reloaded in TC with each build. Make sure that all the dll’s
    the test dll references are available either in the GAC or in the
    dll’s folder.
    - NUnit works the same as MSTest, it just
    requires that NUnit be installed on the machine and that tests
    implements the NUnit framework rather than the MS testing framework.
    A little more work from a testing perspective since the code isn’t
    autogenerated by visual studio correctly, but MSTest isn’t
    stand alone so if the test isn’t being run locally, then
    visual studio needs to be loaded on the testing machine.
  • Unexpected windows:
    - automatic handling only works on modal windows (
    windows that deny access to where TC is trying to get ).
    - TC will press the default button after 10 seconds, then press Esc after
    10 more seconds, then Kill window after 10 more seconds.
    - settings and times can be changed both in code and in the project
    options area.
  • MAPI.SendMail can be used in the error even t handler to send immediate
    notifications of errors to an email, rather than waiting until the
    test has completed before finding out what went wrong. Good for
    really long running tests.
  • TC Engine can be used inside of .NET if you want to integrate it into
    your application.
  • Builtin.RebootAndContinue
    will reboot the machine the test is running on and start the test
    back up when the machine comes backup. Application variables will
    need to be used to make sure the test starts back up at the correct
    place.
  • Web accessibility checkpoint checks a webpage for compliance of various
    things, including government compliance which is very strict. (e.g.
    alt tag, working links, etc…)
  • IEFrame(x), the x is the index of the tabs of a browser. Good idea to keep all
    tests written against the first tab so issues don’t come into
    play of the tab disappearing or causing other tabs to crash.
  • NativeWebObject.Find only works on valid html tags and attributes. It won’t parse
    custom tags ( that might be used for CSS ) and recognize them as
    anything other than pure text.
  • Web testing, quick tips” in the help index is a nice place to get
    a quick overview of a bunch of items that can help when performing
    webtesting. It’s the only copy of this list.
  • Web Services:
    - TC can build a test for a web service from a WSDL. It
    creates the object and some dummy data so the framework is auto
    generated for you.
    - Web service checkpoint, allows you to test a
    specific method in a WSDL. While the returning object is created by
    TC, the data in the object needs to be either specified or loaded in
    code from a data store.
    - Set a variable equal to a WSDL call and
    it will be set to the return value. If the return value exists and
    is an object then intellisense will function.
    - be sure to check if exists on calls to make sure the web service call worked.
  • Load Testing:
    - By default .NET will turn off caching of things like
    javascript when compiled in Debug mode.
    - If perfect performance
    tests of cached javascript are required then the assembly will need
    to be compiled in release mode.
    - remote agent, can send requests
    through other machines for load testing. Add the other machines ip
    address under the Stations list to facilitate this.
    - the remote
    agent can also simulate requests from all sorts of browsers at
    almost any given connection speed. Not a replacement for actual
    testing on a given device though.
    - remote agent and test execute
    can be run on the same machine at the same time, so it is possible
    to have a machine executing tests at the same time it is being used
    as a node for a load test.
  • Distributed Testing:
    - for windows applications uses the “network
    testing” area of TC.
    - When adding a new project task, the
    “path” is the location of the project file (.mds) on the
    destination ( not host ) machine. The “test” option is
    the name of the function to run, by default it is ‘main’
    ( or whatever is set as the default in the project ).
    - you can
    ‘verify’ both hosts and tasks to be sure they exist and
    are available and correct before starting the test. This helps to
    be sure that the basic stuff is covered so you don’t get
    things erroring out before the test even starts.
    - distributed
    testing uses port 9090.
  • TC 6.5 fixed the issue of not supporting Order By and Where clauses in
    query design view. Before it would erase them if you added them
    after the fact then opened up the design window.
  • The only way to use shared connection strings for dbtables is to put one
    in code then in code set all the dbtables connection strings.

  • Code Editor, can be accessed through the code templates in the project
    options. It allows you to write code snippets for reuse later. Put
    a Pipe where you want the cursor to be placed when the snippet is
    inserted.
Questions:
  • Q:
    The testing tool doesn’t allow more than one instance of
    itself to run on the machine making it difficult to use code between
    the Dev and Prod test environments we have setup.
    A: TC is a COM
    server which is the main reason why multiple instances can’t
    run. Try using project level variables ( whose values can be passed
    via command line if desired ) to determine if the test is running in
    Prod or Dev.
    #ifdef Prod
  • Q:
    When viewing logs with IE and test execute my log files are
    frequently so big that it hangs the browser. Can I set it so it
    only shows errors on initial open?
    A: javascript isn’t very
    involved in the rendering of this page, it is all XML and XSL. Try
    using an XML/XSL reader instead of IE. Opening a different app for
    log reading isn’t possible in Test execute, but IE can be
    turned off from displaying when the test is done.
  • Q:
    What is the point of locking events if you still get a “Locked
    events” drilldown line in the log?
    A: Locking events does
    remove them unless an error occurs, then it shows the last 20 or so.
  • Q:
    Intellisense seems to barely function at best in C#, are the other
    languages better?
    A: Intellisense doen’t work on variants,
    a specific object must be declared.
  • Q:
    Using Excel to drive tests is kind of cumbersome and unusual at
    times. Is there an easy way of simply opening the file and looping
    through the data?
    A: DDT.CurrentDriver.Next in a loop or assigned
    variable.Next allows for a single pass through the file. Closing and
    reopening the connection allows for multiple passes.
  • Q:
    Can you move to the first record to start a manual loop over without
    closing down the connection?
    A: No, currently there is no
    movefirst method and no plan to add one.
  • Q:
    You can have dbtables either compare themselves or update themselves
    using the “Compare” method. But what if you want to do
    one of the other at the dbtable level rather than the project suite
    level?
    A: It’s an all or nothing approach. If you need to
    loop through a current data table it might be a better option to use
    DDT.ADODriver.
  • Q:
    Is there a way to set the macro recorder so it records slower yet
    more robust script by using search and find functions rather than
    pathing everything out?
    A: No, but try named mapping, it will
    allow for faster code and simple remapping even if it isn’t
    quite as robust as find.
  • Q:
    “Keys” doesn’t seem to support any keys that
    aren’t on the standard English keyboard. Why since they are
    just other ASCII symbols?
    A: x
  • Q:
    I haven’t tried all the ADO query types, but CreateADOQuery()
    doesn’t seem capable of simply executing a statement without
    returning a resultset, it errors out; why?
    A: Use the straight
    ADO driver with ExecuteNonScalar. TC implements most of the data
    connection options that .NET does so if something is available in
    .NET it probably will be in TC.
  • Q:
    The exception object doesn’t seem to have more than a basic
    description of an error. Am I missing something ( line number, code
    snippet, etc. ) ?
    A: No, since it isn’t a compiled language
    there isn’t much information available to the exception
    object. This can be worked around by implementing your own
    framework with inheritance that does its own error handling and
    produces better errors.
  • Q:
    ToURL doesn’t behave like user input, it doesn’t first
    change the url, so if the destination page fails, you still have the
    original url. This makes troubleshooting more difficult, so I made
    my own ToURL functions. Why does it behave like this?
    A: ToURL
    is faster the way they have it as it makes a direct http request
    bypassing the GUI until it is time for the page to load.
    Implementing a custom function work around is currently the best
    option.
  • Q:
    I couldn’t find a way to “select” a listbox item
    by its value, arguable one of the most important items, am I missing
    something?
    A: No, there is currently no function for that. The
    best workaround is to write your own using the IndexOf function.
  • Q:
    The NativeWebObject’s find and the findall methods are very
    slow, especially with large DOMs. Is there a way for me to write a
    javascript function to return the desired object or array from some
    input?
    A: The Add method is in the DOM object for inserting code,
    it can be used for this purpose.
  • Q:
    What is the AQA development cycle and rev release dates?
    A: Every
    3 months.
  • Q:
    Is there a way to set the macro recorder so it records slower yet
    more robust scripts by using search and find functions rather than
    pathing everything out?
    A: Not yet, but it is being worked on.
  • Q:
    You can compare a database against stored values but can you update
    the database with them? In other words can you reset the data in the
    database?
    Q: automated build studio can restore an entire
    database. However the only way to restore a dbtable is to loop
    through each record in code and update the database with it.
  • Q:
    When creating a new unit the code templates for each language appear
    to be different. Can the templates be changed? Why are they
    different?
    A: They are different because some languages don’t
    support all the features that others do, so each template is
    tailored to give the best jump start for a particular language. The
    templates themselves can’t be modified, but Ctrl+J will bring
    up the template creator which can be used to create code snippets.
  • Q:
    There is a serious lack of documentation of the language features
    which has made progress much slower than it otherwise would have
    been. Any other places I can go?
    A: Sqaforums.com is a great
    reference for the test complete software, probably better than aqa’s
    site itself. Sqaforums is monitored by aqa staff. Also
    Testcompletetv is a website that has short video tutorials covering
    various things in test complete.
  • Q:
    TestComplete seems to have problems finding hyperlinks by HTML ID;
    it seems testcomplete occasionally overwrites the HTML ID with its
    own unique ID. Why is this?
    A: TC doesn’t overwrite any
    HTML Dom objects or attributes. Check the code and see if you have
    a mistake.
  • Q:
    Is there any major efforts to improve the TC tool, including
    documentation?
    A: There is an advanced book coming out in January
    which will be free to download.
    There is a custom script plugin
    framework in the works as well.
  • Q:
    When a variable is used in a method without first declaring it, what
    is the scope of that variable?
    A: it will be a local variant
    variable.
  • Q:
    AQA’s news reader forums don’t have a decent search, are
    they planning on one?
    A: not that I am aware of, however try
    sqaforums.com, it’s a better place for answers anyway.
  • Q:
    Is “TestComplete made easy” the only book you know of
    covering Test Complete?
    A: Yes, until the advanced book comes out
    in January.
  • Q:
    If I want to store sql queries in text files or XML documents rather
    than inline in code, can I easily do that?
    A: No, the
    functionality isn’t currently built-in, the closest thing
    would be storing them in an excel document and loop through that;
    which has its own issues.
  • Q:
    It’s very annoying having to enter so many case sensitive
    registration details, but it’s even worse when the “check
    for updates” link won’t log me into the site.
    A: no
    plans to change the registration process, however the link to log
    into the site is suppose to auto log you in with your current
    registration, it’s just currently broken.
  • Q:
    Is there a way to programmatically refresh the data stored in a TC
    dbtable?
    A: yes, everything that can be done through the GUI can
    be done in code. Would have to open the dbtable in code and rerun
    the query.
  • Q:
    can I trade in the second copy of the TestComplete book we now own
    for a copy of the new advanced version when it comes out?
    A: no,
    but the instructor said he would send me one when it came out.
  • Q:
    Can I change the default app TestExecute uses to view logs when the
    test completes?
    A: No.

Questions I couldn’t get answered due to no laptop:
  • Q:
    On some of my tests, one in particular, TC shuts itself down after
    finishing the test; no error long, no errors, it just closes down
    “Japa Tap Prod ‘User Management’”. Any idea
    why?
    A: try bringing up with AQA team.
  • Q:
    Global variable to TestedApps.IExplorer.run().IEFrame(0) didn’t
    always hold the correct and original value. Why?
  • Q:
    Even when NativeWebObject.Find sees an object, sometimes I can’t
    Find it using UniqueID.
  • Q:
    When TestComplete has to deal with a popup window (i.e. a file
    download) it seems to mess it up internally so things no longer
    function as expected. Any ideas?
  • Q:
    When running a bunch of tests batched up TestComplete will often
    error out on the later tests, even though each test starts up a new
    instance of IE and closes it down when it is done. Why?
    A: don’t
    know, but might try checking to see if the IE process is really
    closed and killing it if it isn’t.
  • Q:
    “Keys” function doesn’t always work correctly.
    Sometimes it prepends data instead of erasing it with “^a”.
    Why?
    A: don’t know, haven’t seen that problem before.
  • Q:
    Sometimes it misses the first field on my PLT form even though the
    field is looked for in code?
    A: check to be sure you are really
    looking for the field. Make sure the field isn’t locked or
    read only.
AQTime
  • AQTime
    checks speed of code and finds memory leaks.
Test Recorder
  • Test
    Recorder can be built into your application so users can record
    tests if they are having problems. There is generally a registry
    setting to allow users to turn on and off the test menu so they
    don’t have to see it unless support is requesting a recorded
    test of their issue.
Automated Build Studio:
  • Seems worth checking out.
  • Less than $200.
  • Can ftp, dts, run macros, has logic paths, and much more.
  • Runs as a service and can even log in a user to start up a test at a
    specific time if needed.
Visual Studio
  • Unit test creation, right click on a method to access the built-in auto
    generate functions.
  • CPU window allows you to see the memory certain code is being run in and
    how it is being run. Can be a major hacker tool.
  • Functions returning string pointers are very hackable as the pointer can be
    changed to point to malicious code or the terminator ( typically a
    \n ) can be removed and malicious code inserted. Ints and objects
    don’t have this problem and aren’t as vulnerable.
  • The US government doesn’t allow any functions in code it runs to
    return string pointers.

Friday, November 21, 2008

List of installed packages in linux

I am using CentOS and was wondering how to get a list of all packages installed on my system:

I found a site: Here

which claims
rpm -qa | sort | less

does the trick, however I am unable to find a package I am pretty sure is installed so I am not sure if their is a better method.