Skip navigation.
Home
The QNX Community Portal

View topic - Python-2.4.1 and sockets

Python-2.4.1 and sockets

Read-only archive of qnx.porting (Discuss migrating applications from QNX4, VxWorks, PSOS, or Linux to QNX Neutrino) at inn.qnx.com

Python-2.4.1 and sockets

Postby lyndseyf » Wed Jul 27, 2005 10:57 pm

I'm having difficulties getting Python-2.4.1 set up on QNX 6.3. Reading the older Python2.4 postings helped and using the QNXconf.sh script I was able to get most everything installed. Everything seems to go well (I can run my old Python scripts, use the command line, etc) until I try to install another application (that relies on Python). During its "./setup.py build" step, I eventually get the following:

Code: Select all
File "/usr/local/lib/python2.4/socket.py", line 45, in ?
    import _socket
ImportError: No module named _socket


I can see from the Python-2.4.1 configuration output that the check for sys/socket.h usability passes and I have tried to edit the Modules/Setup file to get things working but I don't seem to be making any progress. I also tried editing the QNXconf.sh to add '--enable-sockets' to the arguments to the configure script with no luck. I can imagine that I don't actually have all of 2.4.1 happy yet since even though I succeed in getting through the make/make install steps, there were numerous warnings about parsing errors in files. Has anyone overcome the socket issue? (Or, am I asking a question about fixing a symptom and not the real cause?)

Thanks in advance,
Lyndsey
lyndseyf
New Member
 
Posts: 9
Joined: Wed Jul 27, 2005 9:27 pm

Re: Python-2.4.1 and sockets

Postby Steinhoff » Thu Jul 28, 2005 10:05 am

lyndseyf wrote:
I'm having difficulties getting Python-2.4.1 set up on QNX 6.3.
Reading the older Python2.4 postings helped and using the QNXconf.sh
script I was able to get most everything installed. Everything seems
to go well (I can run my old Python scripts, use the command line,
etc) until I try to install another application (that relies on
Python). During its "./setup.py build" step, I eventually get the
following:

File
"/usr/local/lib/python2.4/socket.py", line 45, in ?
import _socket
ImportError: No module named _socket

I can see from the Python-2.4.1 configuration output that the check
for sys/socket.h usability passes and I have tried to edit the
Modules/Setup file to get things working but I don't seem to be
making any progress. I also tried editing the QNXconf.sh to add
'--enable-sockets' to the arguments to the configure script with no
luck. I can imagine that I don't actually have all of 2.4.1 happy
yet since even though I succeed in getting through the make/make
install

AFAIK ... the instalation steps are make, python setup.py build, python
setup.py install. With this sequence I saw no problems with the socket
extension.

--Armin


steps, there were numerous warnings about parsing errors in
files. Has anyone overcome the socket issue? (Or, am I asking a
question about fixing a symptom and not the real cause?)

Thanks in advance,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

re:Python-2.4.1 and sockets

Postby lyndseyf » Thu Jul 28, 2005 3:33 pm

Not quite. My problems with the extension don't come up in the python installation. It's when I try to install the other application (which, by the way, doesn't have a makefile- only the setup.py) that it complains about a lack of the _socket module. I did try to reinstall python and follow the steps you gave but that didn't help me much (perhaps I misinterpreted what you meant?) Any other ideas?

Thanks in advance,
Lyndsey
lyndseyf
New Member
 
Posts: 9
Joined: Wed Jul 27, 2005 9:27 pm

Re: Python-2.4.1 and sockets

Postby Steinhoff » Fri Jul 29, 2005 9:55 am

lyndseyf wrote:
Not quite. My problems with the extension don't come up in the python
installation. It's when I try to install the other application
(which, by the way, doesn't have a makefile- only the setup.py) that
it complains about a lack of the _socket module.

_socket.so should be in /usr/local/lib/python2.4/lib-dynload
(default installation dir ..)

I did try to
reinstall python and follow the steps you gave but that didn't help
me much (perhaps I misinterpreted what you meant?) Any other ideas?

pyhon
import sys
sys.path

Is the path above included? What about LD_LIBRARY_PATH ?

--Armin


Thanks in advance,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

Re: Python-2.4.1 and sockets

Postby Steinhoff » Fri Jul 29, 2005 11:32 am

lyndseyf wrote:
Not quite. My problems with the extension don't come up in the python
installation. It's when I try to install the other application
(which, by the way, doesn't have a makefile- only the setup.py) that
it complains about a lack of the _socket module. I did try to
reinstall python and follow the steps you gave but that didn't help
me much (perhaps I misinterpreted what you meant?) Any other ideas?

The socketmodule.c must be modified! Here is a short HACK!

Change getaddrinfo.c and add:
--------------------------------
/*
* "#ifdef FAITH" part is local hack for supporting IPv4-v6 translator.
*
* Issues to be discussed:
* - Thread safe-ness must be checked.
* - Return values. There are nonstandard return values defined and used
* in the source code. This is because RFC2133 is silent about which
error
* code must be returned for which situation.
* - PF_UNSPEC case would be handled in getipnodebyname() with the
AI_ALL flag.
*/
// qnx specific modification

#ifdef __QNXNTO__
typedef unsigned char uchar_t;
typedef unsigned short ushort_t;
typedef unsigned int uint_t;
typedef unsigned long ulong_t;
typedef unsigned char u_char;
typedef unsigned short u_short;
typedef unsigned int u_int;
typedef unsigned long u_long;
typedef unsigned long fixpt_t;
typedef unsigned short sel_t;
#endif

Modify socketmodule.c and add:


/* Non-MS WINDOWS includes */

#ifdef __QNXNTO__
#include <sys/socket.h>
#undef _XOPEN_SOURCE
#endif

# include <netdb.h>

#ifdef __QNXNTO__
#define _XOPEN_SOURCE
#endif

That's all!

--Armin


Thanks in advance,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

Re: Python-2.4.1 and sockets

Postby Steinhoff » Fri Jul 29, 2005 12:07 pm

Armin Steinhoff wrote:
lyndseyf wrote:

Not quite. My problems with the extension don't come up in the python
installation. It's when I try to install the other application
(which, by the way, doesn't have a makefile- only the setup.py) that
it complains about a lack of the _socket module. I did try to
reinstall python and follow the steps you gave but that didn't help
me much (perhaps I misinterpreted what you meant?) Any other ideas?


The socketmodule.c must be modified! Here is a short HACK!

Sorry ... you have also to modify addrinfo.h in that way:


#ifndef HAVE_ADDRINFO
#ifndef __QNXNTO__
struct addrinfo {
int ai_flags; /* AI_PASSIVE, AI_CANONNAME */
int ai_family; /* PF_xxx */
int ai_socktype; /* SOCK_xxx */
int ai_protocol; /* 0 or IPPROTO_xxx for IPv4 and IPv6 */
size_t ai_addrlen; /* length of ai_addr */
char *ai_canonname; /* canonical name for hostname */
struct sockaddr *ai_addr; /* binary address */
struct addrinfo *ai_next; /* next structure in linked list */
};
#endif
#endif /* !HAVE_ADDRINFO */


Change getaddrinfo.c and add:
--------------------------------
/*
* "#ifdef FAITH" part is local hack for supporting IPv4-v6 translator.
*
* Issues to be discussed:
* - Thread safe-ness must be checked.
* - Return values. There are nonstandard return values defined and used
* in the source code. This is because RFC2133 is silent about which
error
* code must be returned for which situation.
* - PF_UNSPEC case would be handled in getipnodebyname() with the
AI_ALL flag.
*/
// qnx specific modification

#ifdef __QNXNTO__
typedef unsigned char uchar_t;
typedef unsigned short ushort_t;
typedef unsigned int uint_t;
typedef unsigned long ulong_t;
typedef unsigned char u_char;
typedef unsigned short u_short;
typedef unsigned int u_int;
typedef unsigned long u_long;
typedef unsigned long fixpt_t;
typedef unsigned short sel_t;
#endif

Modify socketmodule.c and add:


/* Non-MS WINDOWS includes */

#ifdef __QNXNTO__
#include <sys/socket.h
#undef _XOPEN_SOURCE
#endif

# include <netdb.h

#ifdef __QNXNTO__
#define _XOPEN_SOURCE
#endif

That's all!

--Armin



Thanks in advance,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

Thank you!

Postby lyndseyf » Fri Jul 29, 2005 4:05 pm

Thanks much! That did the trick and I'd have spent days trying to figure all that out.

Thanks,
Lyndsey
lyndseyf
New Member
 
Posts: 9
Joined: Wed Jul 27, 2005 9:27 pm

Re: Thank you!

Postby Steinhoff » Sun Jul 31, 2005 6:14 am

lyndseyf wrote:
Thanks much! That did the trick and I'd have spent
days trying to figure all that out.

Your are welcome. I will try to put the diff files on the pyqnx page.

--Armin

Thanks,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

And now for something completely different (sort of)

Postby lyndseyf » Wed Aug 03, 2005 10:39 pm

Okay, thanks to your help I was able to get my sockets up and running (thanks again!) but now I'm confused on something else (not wholly unrelated). This time I'm fighting with Python's os.fork(). My other application (which is installed, finally :)) uses it in the traditional
Code: Select all
pid = os.fork()
if pid == 0:
       .....
else:
       .....

sense and it complains through a raised OSError that there's a function not implemented (I think its Errno 89). I've tried messing around with other variations of fork like os.forkpty() and pty.fork() but didn't have much luck. Some one suggested I use spawn() but I don't think that spawn will do what the program is expecting (I gave it a half-hearted attempt but haven't tried to force it yet). I've heard that with QNX, "fork is only suppored for a single threaded applications" but that doesn't quite make enough sense to me to help (I'm mostly a newbie). Is there a trick to getting the forks up and running or should I start looking into other methods?

Thanks again and in advance,
Lyndsey
lyndseyf
New Member
 
Posts: 9
Joined: Wed Jul 27, 2005 9:27 pm

re:Python-2.4.1 and sockets

Postby lyndseyf » Wed Aug 03, 2005 11:09 pm

...it would probably help if I gave more details (*sigh* look before I post). For some reason, I can get the os.fork to "work" when I use the Python command prompt but I get some odd (to me) output. I have a simple test script I was playing with that goes something like
Code: Select all
import os
pid = os.fork()
if (0 == pid):
      print "child:",pid
if not (0 == pid):
      print "Parent:",pid
print "Out",pid
from which I get
Code: Select all
Parent: 5458
child: 0
Out 5458
Out 0
sem_wait: >>> Invalid argument
>>>
That looks to me like the parent isn't waiting correctly for the child process but I don't know where to look to fix it. I'm assuming the screwy output is what's causing my other program to complain.

Thanks in advance
Lyndsey
lyndseyf
New Member
 
Posts: 9
Joined: Wed Jul 27, 2005 9:27 pm

Re: Python-2.4.1 and sockets

Postby Steinhoff » Thu Aug 04, 2005 12:02 pm

lyndseyf wrote:
...it would probably help if I gave more details (*sigh* look before I
post). For some reason, I can get the os.fork to "work" when I use
the Python command prompt but I get some odd (to me) output. I have
a simple test script I was playing with that goes something like
import os
pid = os.fork()
if (0 == pid):
print "child:",pid
if not (0 == pid):
print "Parent:",pid
print "Out",pid from which I get
[code:1:e4abb9dac0]Parent: 5458
child: 0
Out 5458
Out 0
sem_wait: >>> Invalid argument

There are in general some problems with the 'Global Interpreter Lock'
(GIL). Python is using for the GIL sem_yxz() calls in order to build
critical sections ... but the recursive use of semaphores are not
supported by QNX6.

--Armin

http://www.steinhoff-automation.com

EtherCAT for QNX6.3 now released


[/code:1:e4abb9dac0] That looks to me like the parent

isn't waiting correctly for the child process but I don't know where
to look to fix it. I'm assuming the screwy output is what's causing
my other program to complain.

Thanks in advance
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

Re: And now for something completely different (sort of)

Postby Steinhoff » Thu Aug 04, 2005 12:02 pm

lyndseyf wrote:
Okay, thanks to your help I was able to get my sockets up and running
(thanks again!) but now I'm confused on something else (not wholly
unrelated). This time I'm fighting with Python's os.fork(). My
other application (which is installed, finally :)) uses it in the
traditional pid = os.fork()
if pid == 0:
.....
else:
.....
sense and it complains through a raised OSError that there's a
function not implemented (I think its Errno 89). I've tried messing
around with other variations of fork like os.forkpty() and pty.fork()
but didn't have much luck. Some one suggested I use spawn() but I
don't think that spawn will do what the program is expecting (I gave
it a half-hearted attempt but haven't tried to force it yet). I've
heard that with QNX, "fork is only suppored for a single threaded
applications" but that doesn't quite make enough sense to me to help
(I'm mostly a newbie). Is there a trick to getting the forks up and
running or should I start looking into other methods?

Yes, that's a knowen problem with the fork and QNX6.

Try to rebuild Python without thread support ... this will also avoid
the problems with the GIL.

--Armin


Thanks again and in advance,
Lyndsey
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am

Re: And now for something completely different (sort of)

Postby John Nagle » Thu Aug 18, 2005 7:27 am

Armin Steinhoff wrote:

lyndseyf wrote:

Okay, thanks to your help I was able to get my sockets up and running
(thanks again!) but now I'm confused on something else (not wholly
unrelated). This time I'm fighting with Python's os.fork(). My
other application (which is installed, finally :)) uses it in the
traditional pid = os.fork()
if pid == 0:
.....
else:
.....
sense and it complains through a raised OSError that there's a
function not implemented (I think its Errno 89). I've tried messing
around with other variations of fork like os.forkpty() and pty.fork()
but didn't have much luck. Some one suggested I use spawn() but I
don't think that spawn will do what the program is expecting (I gave
it a half-hearted attempt but haven't tried to force it yet). I've
heard that with QNX, "fork is only suppored for a single threaded
applications" but that doesn't quite make enough sense to me to help
(I'm mostly a newbie). Is there a trick to getting the forks up and
running or should I start looking into other methods?


Yes, that's a knowen problem with the fork and QNX6.

Try to rebuild Python without thread support ... this will also avoid
the problems with the GIL.

--Armin

It's not really a bug. It's a design question. What do you
want "fork" to do in a multithreaded program? Fork all the
threads? (Sun's Solaris actually did that until Solaris 9.)
Make all the other threads exit? Some versions of UNIX do allow
"fork" in multithreaded programs, duplicating only the forking
thread, but this creates a race condition while memory
is being duplicated.

There's a good discussion of this here:

http://docs.sun.com/app/docs/doc/816-51 ... pjq?a=view

You can call "spawn" in a multithreaded program, which makes
more sense than calling "fork".

John Nagle
John Nagle
 

Re: And now for something completely different (sort of)

Postby Steinhoff » Fri Aug 19, 2005 11:31 am

An interesting approach to avoid the GIL is also:
http://poshmodule.sourceforge.net/

--Armin


John Nagle wrote:
Armin Steinhoff wrote:

lyndseyf wrote:

Okay, thanks to your help I was able to get my sockets up and running
(thanks again!) but now I'm confused on something else (not wholly
unrelated). This time I'm fighting with Python's os.fork(). My
other application (which is installed, finally :)) uses it in the
traditional pid = os.fork()
if pid == 0:
.....
else:
.....
sense and it complains through a raised OSError that there's a
function not implemented (I think its Errno 89). I've tried messing
around with other variations of fork like os.forkpty() and pty.fork()
but didn't have much luck. Some one suggested I use spawn() but I
don't think that spawn will do what the program is expecting (I gave
it a half-hearted attempt but haven't tried to force it yet). I've
heard that with QNX, "fork is only suppored for a single threaded
applications" but that doesn't quite make enough sense to me to help
(I'm mostly a newbie). Is there a trick to getting the forks up and
running or should I start looking into other methods?



Yes, that's a knowen problem with the fork and QNX6.

Try to rebuild Python without thread support ... this will also avoid
the problems with the GIL.

--Armin


It's not really a bug. It's a design question. What do you
want "fork" to do in a multithreaded program? Fork all the
threads? (Sun's Solaris actually did that until Solaris 9.)
Make all the other threads exit? Some versions of UNIX do allow
"fork" in multithreaded programs, duplicating only the forking
thread, but this creates a race condition while memory
is being duplicated.

There's a good discussion of this here:

http://docs.sun.com/app/docs/doc/816-51 ... pjq?a=view

You can call "spawn" in a multithreaded program, which makes
more sense than calling "fork".

John Nagle
Steinhoff
QNX Master
 
Posts: 279
Joined: Tue Sep 03, 2002 11:56 am


Return to qnx.porting

Who is online

Users browsing this forum: No registered users and 1 guest