Saturday, July 21, 2012

SOLVED: How to Install Netgear WG311 under Ubuntu 12.04

Put Netgear WG311 in box. turn on box. follow directions at https://help.ubuntu.com/community/WifiDocs/Device/Netgear_WG311_v3 BUT: No matter how many times I rebooted, the sudo modprobe ndiswrapper kept giving me the message FATAL. Solved via:
kevin@odin:~$ sudo modprobe ndiswrapper
[sudo] password for kevin: 
FATAL: Module ndiswrapper not found.


kevin@odin:~$ sudo apt-cache search dkms
... (omitting extra stuff)....
dkms - Dynamic Kernel Module Support Framework
ndiswrapper-dkms - Source for the ndiswrapper Linux kernel module (DKMS)
... (omitting extra stuff)....

kevin@odin:~$ sudo apt-get install dkms
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  fakeroot patch
Suggested packages:
  diffutils-doc
The following NEW packages will be installed:
  dkms fakeroot patch
0 upgraded, 3 newly installed, 0 to remove and 9 not upgraded.
Need to get 247 kB of archives.
After this operation, 890 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Get:1 http://mirror.anl.gov/pub/ubuntu/ precise/main patch i386 2.6.1-3 [86.0 kB]
Get:2 http://mirror.anl.gov/pub/ubuntu/ precise/main dkms all 2.2.0.3-1ubuntu3 [73.1 kB]
Get:3 http://mirror.anl.gov/pub/ubuntu/ precise/main fakeroot i386 1.18.2-1 [87.9 kB]
Fetched 247 kB in 0s (436 kB/s)
Selecting previously unselected package patch.
(Reading database ... 143936 files and directories currently installed.)
Unpacking patch (from .../patch_2.6.1-3_i386.deb) ...
Selecting previously unselected package dkms.
Unpacking dkms (from .../dkms_2.2.0.3-1ubuntu3_all.deb) ...
Selecting previously unselected package fakeroot.
Unpacking fakeroot (from .../fakeroot_1.18.2-1_i386.deb) ...
Processing triggers for man-db ...
Setting up patch (2.6.1-3) ...
Setting up dkms (2.2.0.3-1ubuntu3) ...
Setting up fakeroot (1.18.2-1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode.

kevin@odin:~$ sudo apt-get install dkms ndiswrapper-dkms
Reading package lists... Done
Building dependency tree       
Reading state information... Done
dkms is already the newest version.
The following NEW packages will be installed:
  ndiswrapper-dkms
0 upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 176 kB of archives.
After this operation, 778 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Get:1 http://mirror.anl.gov/pub/ubuntu/ precise/universe ndiswrapper-dkms all 1.57-1ubuntu1 [176 kB]
Fetched 176 kB in 0s (285 kB/s)      
Selecting previously unselected package ndiswrapper-dkms.
(Reading database ... 144030 files and directories currently installed.)
Unpacking ndiswrapper-dkms (from .../ndiswrapper-dkms_1.57-1ubuntu1_all.deb) ...
Setting up ndiswrapper-dkms (1.57-1ubuntu1) ...
Loading new ndiswrapper-1.57 DKMS files...
First Installation: checking all kernels...
Building only for 3.2.0-23-generic-pae
Building initial module for 3.2.0-23-generic-pae
Done.

ndiswrapper:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/3.2.0-23-generic-pae/updates/dkms/

depmod.......

DKMS: install completed.
kevin@odin:~$ iwconfig
lo        no wireless extensions.

eth0      no wireless extensions.

kevin@odin:~$ sudo modprobe ndiswrapper
kevin@odin:~$ iwconfig
lo        no wireless extensions.

wlan0     IEEE 802.11g  ESSID:off/any  
          Mode:Managed  Channel:0  Access Point: Not-Associated   
          Bit Rate:1 Mb/s   Sensitivity=-200 dBm  
          RTS thr=2346 B   Fragment thr=2346 B   
          Power Management:off
          Link Quality:0  Signal level:0  Noise level:0
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

eth0      no wireless extensions.

Wednesday, July 18, 2012

Just had to create a median function in Python. We compute median values by ordering them and choosing the middle one. I found a routine, but it lacked unittest coverage and the edge case of an empty input array. So, here goes:
import unittest
###############################################################################

def getMedian(numericValues):
    
    theValues = sorted(numericValues)
    if not theValues:
        return None
    
    if len(theValues) % 2 == 1:
        return theValues[(len(theValues)+1)/2-1]
    else:
        lower = theValues[len(theValues)/2-1]
        upper = theValues[len(theValues)/2]
  
        return (float(lower + upper)) / 2  

###############################################################################

if __name__ == '__main__':
    unittest.main()

###############################################################################

class TestCommonUtils(unittest.TestCase):

    def test_trivial(self):
        return True

    def isValid(self, functionName, valueShouldBe, valueIs):
        assert (valueShouldBe == valueIs), "ERROR: Function %s returned value %.6fs, but value should be: %.6f." % (functionName, valueIs, valueShouldBe)  

    def test_getMedian(self):
        self.isValid('getMedian', 2.5,      getMedian([0,1,2,3,4,5]))
        self.isValid('getMedian', 2,        getMedian([0,1,2,3,4]))
        self.isValid('getMedian', 2,        getMedian([3,1,2]))
        self.isValid('getMedian', 3,        getMedian([3,2,3]))
        self.isValid('getMedian', 1.234,    getMedian([1.234, 3.678, -2.467]))
        self.isValid('getMedian', 1.345,    getMedian([1.234, 3.678, 1.456, -2.467]))
        self.isValid('getMedian', 3,        getMedian([3]))
        self.isValid('getMedian', None,     getMedian([]))


$ nosetests util.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Wednesday, July 11, 2012

How to Build ZeroMQ (and Python binding) on AIX

Okay, so I may be stupid. I'm trying to run a Python script on an AIX box. In order to run it, I need a ZeroMQ (see http://zeromq.org). So, I'm trying to compile the zmq library using the standard compile-link-run thing.

BTW, this is AIX 6.1 using the gcc compiler [ TO START WITH ]

First, I had to get the sysadmins to install gcc and g++. then, I kept getting the error:

...
checking whether we are using clang C compiler... no
checking whether we are using gcc >= 4 C compiler... yes
checking whether the C++ compiler works... no
configure: error: Unable to find a working C++ compiler

Ug. Turns out, I didn't have libm installed. Some sysadmin time playing with puppet and NIM later, I have it. Now, I see:

...
checking alloca.h presence... yes
checking for alloca.h... yes
checking whether SOCK_CLOEXEC is supported... no
configure: creating ./config.status
./config.status: line 765: print: command not found
./config.status: line 765: print: command not found
./config.status: line 765: print: command not found
./config.status: line 765: print: command not found
... (times 200 or so)

Ah, fun with vi. Not vim, which tells you line numbers, just vi. Because IBM doesn't know from modern. So, I look up line number and find it's colon-number for go-to-line-number.

I look in the config.status line 765 (with :765), and I see it's referring to $ECHO shell var, I look for 'print' and find $ECHO is defined as:

...
pic_mode='default'
enable_fast_install='yes'
SHELL='/opt/freeware/bin/bash'
ECHO='print -r --'
host_alias=''
host='powerpc-ibm-aix6.1.0.0'
...

ECHO is 'print -r --' ????? Where did that come from?

So, I tried again, doing export ECHO=/usr/bin/echo first. No go. Next, tried with:

ECHO=/usr/bin/echo ./configure --prefix=/destination/directory

No go again.

-- time passes --

to fix this:

  1. in your ~/.profile or .bashrc, define:
    CONFIG_SHELL=/usr/bin/bash
  2. Then, invoke ./configure as follows:
    ${CONFIG_SHELL} ./configure --prefix=...

Continuing, I found out I had to install:

  • gnu automake
  • gnu libtool
  • gnu M4

Now, I'm stuck with:

[myuser@mybox ~/making/zeromq-2.2.0]: make
Making all in src
make[1]: Entering directory `/home/myuser/making/zeromq-2.2.0/src'
make  all-am
make[2]: Entering directory `/home/myuser/making/zeromq-2.2.0/src'
  CXX      libzmq_la-clock.lo
../libtool: line 880: X--tag=CXX: command not found
../libtool: line 913: libtool: ignoring unknown tag : command not found
../libtool: line 880: X--mode=compile: command not found
../libtool: line 1047: *** Warning: inferring the mode of operation is deprecated.: command not found
../libtool: line 1048: *** Future versions of Libtool will require --mode=MODE be specified.: command not found
../libtool: line 1191: Xg++: command not found
../libtool: line 1191: X-DHAVE_CONFIG_H: command not found
../libtool: line 1191: X-I.: command not found
../libtool: line 1191: X-pedantic: command not found
../libtool: line 1191: X-Werror: command not found
../libtool: line 1191: X-Wall: command not found
../libtool: line 1191: X-D_REENTRANT: command not found
../libtool: line 1191: X-D_THREAD_SAFE: command not found
../libtool: line 1191: X-DZMQ_FORCE_POLL: command not found
../libtool: line 1191: X-g: command not found
../libtool: line 1191: X-O2: command not found
../libtool: line 1191: X-MT: command not found
../libtool: line 1191: Xlibzmq_la-clock.lo: command not found
../libtool: line 1191: X-MD: command not found
../libtool: line 1191: X-MP: command not found
../libtool: line 1191: X-MF: command not found
../libtool: line 1191: X.deps/libzmq_la-clock.Tpo: No such file or directory
../libtool: line 1191: X-c: command not found
../libtool: line 1244: Xlibzmq_la-clock.lo: command not found
../libtool: line 1249: libtool: compile: cannot determine name of library object from `': command not found
make[2]: *** [libzmq_la-clock.lo] Error 1
make[2]: Leaving directory `/home/myuser/making/zeromq-2.2.0/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/myuser/making/zeromq-2.2.0/src'
make: *** [all-recursive] Error 1
[myuser@mybox ~/making/zeromq-2.2.0]: 
So, thinking I might not have a valid C/C++ compiler (silly, it should work), I download gcc source 4.2 and try to compile. Error message is:
checking for .preinit_array/.init_array/.fini_array support... no
checking if mkdir takes one argument... no
*** Configuration rs6000-ibm-aix not supported
make[2]: *** [configure-stage1-gcc] Error 1
make[2]: Leaving directory `/home/myuser/making/gcc-4.2.4/gcc-4.2.4'
make[1]: *** [stage1-bubble] Error 2
make[1]: Leaving directory `/home/myuser/making/gcc-4.2.4/gcc-4.2.4'
make: *** [all] Error 2
so out of luck there, too. Ug.

What to do? Punt.

I went to IBM's site and signed up for a free copy of their XLC compiler. I downloaded it and handed it over to the sysadmins, who did their magic to it. It comes in a package-thing that is only installable by IBM's installer, so this requires sysadmins.

Once I had it, I changed my env vars to be:

export PYTHONPATH=/opt/recon/dcm/platforms/AIX/pylibs:/opt/recon/dcm/platforms/AIX/pylibs/lib/python2.6/site-packages:$PYTHONPATH
export PATH=$PATH:/opt/recon/dcm/bin:/opt/recon/dcm/AIX/bin:/opt/recon/dcm/platforms/AIX/pylibs/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/recon/dcm/lib:/opt/recon/dcm/platforms/AIX/lib
export CONFIG_SHELL=/opt/freeware/bin/bash
export CONFIG_ENV_ARGS=/opt/freeware/bin/bash
export PS1="[\u@\H \w]: "

export CONFIG_SHELL=/opt/freeware/bin/bash
export CONFIG_ENV_ARGS=/opt/freeware/bin/bash
export CC=cc
export CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 -D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"
export CXX=xlC
export CXXFLAGS=$CFLAGS
export F77=xlf
export FFLAGS="-qmaxmem=16384 -O -I/opt/freeware/include"
export LD=ld
export LDFLAGS="-L/opt/freeware/lib -Wl,-bmaxdata:0x80000000"
export PATH=/opt/recon/dcm/platforms/AIX/bin:/usr/bin:/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/vac/bin:/usr/vacpp/bin:/usr/ccs/bin:/usr/dt/bin:/usr/opt/perl5/bin:/opt/freeware/bin:/opt/freeware/sbin:/usr/local/bin:/usr/lib/instl
then LOGGED OUT and RELOGGED-IN to pick up the env changes, and proceeded.

I then rebuilt libtool (because the aix version was way old) by doing the standard

${CONFIG_SHELL} ./configure --prefix=...
make
make check
make install
(and it worked!),

I re-untarred the zeromq library, and did:

${CONFIG_SHELL} ./configure --prefix=...
make
make check
but this failed. Problem was
make[1]: Entering directory `/opt/recon/making/zeromq-2.2.0/tests'
make  check-TESTS
make[2]: Entering directory `/opt/recon/making/zeromq-2.2.0/tests'
Could not load program /opt/recon/making/zeromq-2.2.0/tests/.libs/lt-test_pair_inproc:
Could not load module /opt/recon/dcm/platforms/AIX/lib/libzmq.a(libzmq.so.1).
        Dependent module /opt/recon/dcm/platforms/AIX/lib/libcrypto.a(libcrypto.so.1.0.0) could not be loaded.
        Member libcrypto.so.1.0.0 is not found in archive 
Could not load module lt-test_pair_inproc.
        Dependent module /opt/recon/dcm/platforms/AIX/lib/libzmq.a(libzmq.so.1) could not be loaded.
Could not load module .
FAIL: test_pair_inproc
Could not load program /opt/recon/making/zeromq-2.2.0/tests/.libs/lt-test_pair_tcp:
Could not load module /opt/recon/dcm/platforms/AIX/lib/libzmq.a(libzmq.so.1).
        Dependent module /opt/recon/dcm/platforms/AIX/lib/libcrypto.a(libcrypto.so.1.0.0) could not be loaded.
        Member libcrypto.so.1.0.0 is not found in archive 
Could not load module lt-test_pair_tcp.
...

Long story short, the fix for this is:

We have to add the /opt/freeware/lib to the LIBPATH **** FIRST ****. Otherwise, it can't know to find the libcrypto.so.1.0.0 there. You'll get this same error if /opt/freeware/lib is there but not first in line.
export LIBPATH="/opt/freeware/lib:/opt/freeware/lib64:/usr/lib:/usr/local/lib"
and
export LDFLAGS="-L/opt/freeware/lib -L/opt/freeware/lib64 -Wl,-bmaxdata:0x80000000"
then, success:
make[1]: Entering directory `/opt/recon/making/zeromq-2.2.0/tests'
make  check-TESTS
make[2]: Entering directory `/opt/recon/making/zeromq-2.2.0/tests'
PASS: test_pair_inproc
PASS: test_pair_tcp
PASS: test_reqrep_inproc
PASS: test_reqrep_tcp
PASS: test_hwm
PASS: test_shutdown_stress
PASS: test_pair_ipc
PASS: test_reqrep_ipc
test_timeo running...
PASS: test_timeo
==================
All 9 tests passed
==================
make[2]: Leaving directory `/opt/recon/making/zeromq-2.2.0/tests'

SUCCESS. Next, it's pyzmq.

Too much good luck now, I know this is going to fail soon.

... And I'm right. Ug. ... Okay, so (some time later) I've gotten it to compile. It kept giving me problems with:

    ******************************************
    building 'zmq.core._poll' extension
    cc -DNDEBUG -O -qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 -D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include -I/opt/recon/dcm/platforms/AIX/include -I/opt/recon/dcm/platforms/AIX/include -Izmq/utils -Izmq/core -Izmq/devices -I/opt/freeware/include/python2.6 -c zmq/core/_poll.c -o build/temp.aix-6.1-2.6/zmq/core/_poll.o -Wno-unused-function -Wno-strict-aliasing
    cc: 1501-289 (W) Option -Wno-unused-function was incorrectly specified. The option will be ignored.
    cc: 1501-289 (W) Option -Wno-strict-aliasing was incorrectly specified. The option will be ignored.
    "zmq/utils/zmq_compat.h", line 22.11: 1506-189 (S) Floating-point constant 2.2.0 is not valid.
    "zmq/utils/zmq_compat.h", line 30.11: 1506-189 (S) Floating-point constant 3.0.0 is not valid.
    "zmq/utils/zmq_compat.h", line 70.15: 1506-189 (S) Floating-point constant 3.0.0 is not valid.
    "zmq/utils/zmq_compat.h", line 120.11: 1506-189 (S) Floating-point constant 4.0.0 is not valid.
    "zmq/utils/zmq_compat.h", line 132.15: 1506-189 (S) Floating-point constant 4.0.0 is not valid.
    "zmq/utils/zmq_compat.h", line 147.41: 1506-209 (S) Character constants must end before the end of a line.
    "zmq/utils/zmq_compat.h", line 147.31: 1506-076 (W) Character constant 's fd.hpp)' has more than 4 characters. No more than rightmost 4 characters are used.
    "zmq/utils/zmq_compat.h", line 1.1: 1506-046 (S) Syntax error.
    error: command 'cc' failed with exit status 1
there was nothing about this online. Doomed. ug. So, I peeked in the zmq/utils/zmq_compat.h file to see what it was objecting over. The lines were freaking comments. For some unknown inane stupid reason, the IBM cc (c compiler) objected to perfectly valid comment lines "// new in 3.3.0". So, I know how to edit out comment lines!!! I backed up the file and removed them. Success compiling! I see lots of copies to the prefix directory!

Now, to test: invoke python, import zmq...:

: python
Python 2.6.7 (r267:88850, Nov 22 2011, 20:59:18) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
Traceback (most recent call last):
  File "", line 1, in 
  File "zmq/__init__.py", line 26, in 
    from zmq.utils import initthreads # initialize threads
ImportError: cannot import name initthreads
>>> 
Yuck, but ... memory arises - Ah ha! I've seen this before. This happens BECAUSE I'M IN THE WRONG DIRECTORY. It's picking up something wrong because i"m in the directory I built pyzmq in. So, I go up one dir:
[esm@aixengsb301p ~/making/pyzmq-2.2.0]: cd ..
[esm@aixengsb301p ~/making]: python
Python 2.6.7 (r267:88850, Nov 22 2011, 20:59:18) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
>>> 
[esm@aixengsb301p ~/making]: 

SUCCESS!!!!

Monday, July 02, 2012

Compiling Python with Bz2

Building python from source on RHEL5, I've had some troubles. Python keeps not finding bz2, giving me the complaint:

Python build finished, but the necessary bits to build these modules were not found:
_tkinter           bsddb185           dl              
gdbm               imageop            sunaudiodev     
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Failed to build these modules:
_curses            _curses_panel   _bz2

I already BUILT bz2 and installed it with the right prefix. UG.

FINALLY I found the following hint. Turns out bz2 source compile / make doesn't make everything by default. It just makes the executables bzunzip2, etc., but not the all-important (to python compiling) libbz2_so.

There's a file there, though, you just have to invoke it!

$ cd ~/temp
$ wget http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz
$ tar zxvf bzip2-1.0.5.tar.gz
$ cd bzip2-1.0.5
next, TWO MAKES. INVOKE BOTH MAKEFILES:
$ make -f Makefile-libbz2_so
$ make
then, it works. NOTE: for the rest of the instructions, I like the page: http://rajaseelan.com/2012/01/28/installing-python-2-dot-7-2-on-centos-5-dot-2/