Problem sending classes outside of QNX

bridged with qdn.public.neutrino
Post Reply
Kevin White

Problem sending classes outside of QNX

Post by Kevin White » Wed Jul 30, 2003 5:10 pm

I am trying to get some information regarding how classes get defined and
handled in QNX, in comparison to other OSes. We currently have a problem in
this area, and it is only appearing in one of several OSes that we are
developing in -- QNX.

At any rate, we are utilizing socket communications, and we are sending
information by transmitting a class across the wire. This works rather well
in Windows, VxWorks, and Linux, but on QNX it seems to work much
differently. When we print out all of the data from the class that we are
sending, on the other operating systems, there is 4 bytes of information at
the front of the socket message that seems to be related to the class
information -- if we remove or re-arrange variables or methods within the
class, these numbers change. For the QNX system, these numbers seem to be
close to the end of the socket message, or between class and subclass
boundaries, perhaps. Whatever the case, we can communicate using sockets
within the context of the PC, but when we actually try to talk to another
box (PC, Linux or whatever), the comms fails because the data is not aligned
the same as the other three OSes, and the class information that is being
sent from another operating system is mis-interpreted as data.

Has anyone seen anything like this before, or know how to correct for this?

-Kevin

Mario Charest

Re: Problem sending classes outside of QNX

Post by Mario Charest » Fri Aug 01, 2003 11:22 pm

"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bg8tsh$7mk$1@inn.qnx.com...
I am trying to get some information regarding how classes get defined and
handled in QNX, in comparison to other OSes. We currently have a problem
in
this area, and it is only appearing in one of several OSes that we are
developing in -- QNX.


At any rate, we are utilizing socket communications, and we are sending
information by transmitting a class across the wire. This works rather
well
in Windows, VxWorks, and Linux, but on QNX it seems to work much
differently. When we print out all of the data from the class that we are
sending, on the other operating systems, there is 4 bytes of information
at
the front of the socket message that seems to be related to the class
information -- if we remove or re-arrange variables or methods within the
class, these numbers change. For the QNX system, these numbers seem to be
close to the end of the socket message, or between class and subclass
boundaries, perhaps. Whatever the case, we can communicate using sockets
within the context of the PC, but when we actually try to talk to another
box (PC, Linux or whatever), the comms fails because the data is not
aligned
the same as the other three OSes, and the class information that is being
sent from another operating system is mis-interpreted as data.

Has anyone seen anything like this before, or know how to correct for
this?
I beleive you are asking for trouble. There is no requirement in C++ for
classes to have a specific format. Furthermore this has nothing to do with
the OS but rather with the compiler. If you build a design based on this
your design is broken. You need to serialized the data or have a structure
define in your class and send the structure instead. Class may contains
offset/pointer which become meaningless when sent across the network or even
across processes in a virtual memory system.
-Kevin

Dale Sherwood

Re: Problem sending classes outside of QNX

Post by Dale Sherwood » Mon Aug 04, 2003 10:13 pm

Not to mention the virtual tables included in polymorphic class objects.

Mario, what do you mean by "serialize" the data?

->>>--Dale-Sherwood-->
New York Air Brake Corp.
TDS Group

"Mario Charest" <postmaster@127.0.0.1> wrote in message
news:bges4v$ja8$1@inn.qnx.com...
"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bg8tsh$7mk$1@inn.qnx.com...
[snip]
differently. When we print out all of the data from the class that we
are
sending, on the other operating systems, there is 4 bytes of information
at
the front of the socket message that seems to be related to the class
information -- if we remove or re-arrange variables or methods within
the
class, these numbers change. For the QNX system, these numbers seem to
be
close to the end of the socket message, or between class and subclass
boundaries, perhaps. Whatever the case, we can communicate using
sockets
[snip]
I beleive you are asking for trouble. There is no requirement in C++ for
classes to have a specific format. Furthermore this has nothing to do
with
the OS but rather with the compiler. If you build a design based on this
your design is broken. You need to serialized the data or have a
structure
define in your class and send the structure instead. Class may contains
offset/pointer which become meaningless when sent across the network or
even
across processes in a virtual memory system.


-Kevin

Steve Cobb

Re: Problem sending classes outside of QNX

Post by Steve Cobb » Tue Aug 05, 2003 5:51 am

"Dale Sherwood" <dale.sherwood@SPAM-BITBUCKET-nyab.com> wrote in message
news:bgml66$9sv$1@inn.qnx.com...
Not to mention the virtual tables included in polymorphic class objects.

Mario, what do you mean by "serialize" the data?
Well, I probably shouldn't answer for him :-) But you need to but the
"data" parts of the object into a form that can be "sent" to the other
system. Specifically, the data elements in the structure. Further, if the
object is polymorphic, you must supply whatever information is necessary in
the decomposed data to allow the system on the other end reconstruct the
whole object.

This is probably a good place for XML, or at least a simple ASCII tagged
format. This is especially true if the "object" may be passed between
processors with different "endian" (big vs. little).

This all holds true also if you want to save the object to a file for later
retrieval. Just dumping the whole object as binary data into a file will
lead to all sorts of problems.

Cheers,
Steve

Bill Caroselli

Re: Problem sending classes outside of QNX

Post by Bill Caroselli » Tue Aug 05, 2003 2:00 pm

"Mario Charest" <postmaster@127.0.0.1> wrote in message
news:bges4v$ja8$1@inn.qnx.com...
"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bg8tsh$7mk$1@inn.qnx.com...
I am trying to get some information regarding how classes get defined
and
handled in QNX, in comparison to other OSes. We currently have a
problem
in
this area, and it is only appearing in one of several OSes that we are
developing in -- QNX.


At any rate, we are utilizing socket communications, and we are sending
information by transmitting a class across the wire. This works rather
well
in Windows, VxWorks, and Linux, but on QNX it seems to work much
differently. When we print out all of the data from the class that we
are
sending, on the other operating systems, there is 4 bytes of information
at
the front of the socket message that seems to be related to the class
information -- if we remove or re-arrange variables or methods within
the
class, these numbers change. For the QNX system, these numbers seem to
be
close to the end of the socket message, or between class and subclass
boundaries, perhaps. Whatever the case, we can communicate using
sockets
within the context of the PC, but when we actually try to talk to
another
box (PC, Linux or whatever), the comms fails because the data is not
aligned
the same as the other three OSes, and the class information that is
being
sent from another operating system is mis-interpreted as data.

Has anyone seen anything like this before, or know how to correct for
this?

I beleive you are asking for trouble. There is no requirement in C++ for
classes to have a specific format. Furthermore this has nothing to do
with
the OS but rather with the compiler. If you build a design based on this
your design is broken. You need to serialized the data or have a
structure
define in your class and send the structure instead. Class may contains
offset/pointer which become meaningless when sent across the network or
even
across processes in a virtual memory system.
This is true. Structs should be used to record layouts and message formats.
But if your going to argue C++ legaleese remember, the *ONLY* difference
between a class and a struct is that the contents of a struct is public by
default. The contents of a class are private by default.

You can use a class and honor the rules just like you can use a struct and
break the rules.

Dale Sherwood

Re: Problem sending classes outside of QNX

Post by Dale Sherwood » Tue Aug 05, 2003 4:05 pm

Steve:

"Steve Cobb" <steve_cobb0@yahoo.com> wrote in message
news:bgnft9$qh4$1@inn.qnx.com...
"Dale Sherwood" <dale.sherwood@SPAM-BITBUCKET-nyab.com> wrote in message
news:bgml66$9sv$1@inn.qnx.com...
Not to mention the virtual tables included in polymorphic class objects.

Mario, what do you mean by "serialize" the data?


Well, I probably shouldn't answer for him :-) But you need to but the
"data" parts of the object into a form that can be "sent" to the other
system. Specifically, the data elements in the structure. Further, if
the
object is polymorphic, you must supply whatever information is necessary
in
the decomposed data to allow the system on the other end reconstruct the
whole object.

This is probably a good place for XML, or at least a simple ASCII tagged
format. This is especially true if the "object" may be passed between
processors with different "endian" (big vs. little).
Figured it had to be text, just wasn't familiar with the term "serialize" in
this context. I have since discovered Sun's java.io.Serializable interface,
so I assume the term derives from that.

If ones goal is to *instantiate* objects on another computer instead of
transmit them, something like CORBA might be helpful (but not easy).
This all holds true also if you want to save the object to a file for
later
retrieval. Just dumping the whole object as binary data into a file will
lead to all sorts of problems.
For C++ I like to implement operator<< and operator>> to write and read the
(textual) object state. To do this polymorphically these operators can call
virtual writeObject() and readObject() methods (just don't call them in a
constructor!). For example, the simplest representation of object state is
"member1 member2 member3 ... memberN". Combined with a virtual string
getClass() method, something like "class member1 member2 member3 ...
memberN" might work nicely (as a first approximation--I have not completely
thought this out).

->>>--Dale-Sherwood-->
New York Air Brake Corp.
TDS Group

Kevin White

Re: Problem sending classes outside of QNX

Post by Kevin White » Tue Aug 05, 2003 7:04 pm

So, what I am hearing is that I need to define a structure for my data, but
what is a class -- is it not identical to a structure, except that a class
is private by default and a structure is public? I can understand that
there are some hidden definitions to both of these, and that there may be
alignment problems, but the fact is that it works fine in several different
operating systems, and different platforms (different endian-ness as well).

It would seem to me that this problem is one of two things -- either a
compiler issue, or an issue with QNX internals not adhering to some standard
that these other operating systems have managed to agree upon somehow. I
was hoping that there was some secret compiler setting that would make all
of this a bit more managable, but we have since had to re-write most of
this.

What we had was a nice high-level oo way of dealing with things -- a message
base class, custom messages would inherit from this, and you just send them
out -- lower-level functions handled the rest. The receive side would
receive the base class, if the message id was a match, then cast it into the
appropriate message class, and utilize the contents at will.

I can understand how class information is not applicable outside of certain
scopes, but doesn't a struct have the same information somewhere? And most
importantly, any ideas why this works everywhere else? Being that it works
on three other platforms, I am guessing it is more than luck.

-Kevin

Mario Charest <postmaster@127.0.0.1> wrote in message
news:bges4v$ja8$1@inn.qnx.com...
"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bg8tsh$7mk$1@inn.qnx.com...
I am trying to get some information regarding how classes get defined
and
handled in QNX, in comparison to other OSes. We currently have a
problem
in
this area, and it is only appearing in one of several OSes that we are
developing in -- QNX.


At any rate, we are utilizing socket communications, and we are sending
information by transmitting a class across the wire. This works rather
well
in Windows, VxWorks, and Linux, but on QNX it seems to work much
differently. When we print out all of the data from the class that we
are
sending, on the other operating systems, there is 4 bytes of information
at
the front of the socket message that seems to be related to the class
information -- if we remove or re-arrange variables or methods within
the
class, these numbers change. For the QNX system, these numbers seem to
be
close to the end of the socket message, or between class and subclass
boundaries, perhaps. Whatever the case, we can communicate using
sockets
within the context of the PC, but when we actually try to talk to
another
box (PC, Linux or whatever), the comms fails because the data is not
aligned
the same as the other three OSes, and the class information that is
being
sent from another operating system is mis-interpreted as data.

Has anyone seen anything like this before, or know how to correct for
this?

I beleive you are asking for trouble. There is no requirement in C++ for
classes to have a specific format. Furthermore this has nothing to do
with
the OS but rather with the compiler. If you build a design based on this
your design is broken. You need to serialized the data or have a
structure
define in your class and send the structure instead. Class may contains
offset/pointer which become meaningless when sent across the network or
even
across processes in a virtual memory system.


-Kevin



Dale Sherwood

Re: Problem sending classes outside of QNX

Post by Dale Sherwood » Wed Aug 06, 2003 1:20 am

Kevin:

"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bgouv2$1gt$1@inn.qnx.com...
So, what I am hearing is that I need to define a structure for my data,
but what is a class -- is it not identical to a structure, except that a
class is private by default and a structure is public? I can understand
that there are some hidden definitions to both of these, and that there
may be alignment problems, but the fact is that it works fine in several
different operating systems, and different platforms (different
endian-ness as well).
Just curious: How are you able to send binary objects and not have endian
issues? Is everything a char? Do you define a standard endian format for
larger data types?

[snip]
What we had was a nice high-level oo way of dealing with things -- a
message base class, custom messages would inherit from this, and you
just send them out -- lower-level functions handled the rest. The receive
side would receive the base class, if the message id was a match, then
cast it into the appropriate message class, and utilize the contents at
will.

I can understand how class information is not applicable outside of
certain scopes, but doesn't a struct have the same information somewhere?
And most importantly, any ideas why this works everywhere else? Being
that it works on three other platforms, I am guessing it is more than
luck.
Any virtual methods in your message base class? The reason I ask is that
your original description of the problem made me think of an object that
contains a virtual table (VTABLE). Different compilers might implement the
VTABLE differently, and it is at least conceivable that your other three
compilers locate them outside the objects. If that is the case, then you
indeed have some reimplementing to do.

->>>--Dale-Sherwood-->
New York Air Brake Corp.
TDS Group

Kevin White

Re: Problem sending classes outside of QNX

Post by Kevin White » Wed Aug 06, 2003 6:18 pm

Dale:

Dale Sherwood <dale.sherwood@SPAM-BITBUCKET-nyab.com> wrote in message
news:bgpkgp$f6q$1@inn.qnx.com...
Kevin:

"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bgouv2$1gt$1@inn.qnx.com...
So, what I am hearing is that I need to define a structure for my data,
but what is a class -- is it not identical to a structure, except that a
class is private by default and a structure is public? I can understand
that there are some hidden definitions to both of these, and that there
may be alignment problems, but the fact is that it works fine in several
different operating systems, and different platforms (different
endian-ness as well).

Just curious: How are you able to send binary objects and not have endian
issues? Is everything a char? Do you define a standard endian format for
larger data types?
Actually, we seem to be casting the entire object into an array of
characters, and yes there are a few methods in the base class that are
virtual. Since we are doing things across multiple platforms, we have a set
of swap routines that we use, and do all of that swapping stuff at
just-the-right-time to avoid having problems in the end.
[snip]
What we had was a nice high-level oo way of dealing with things -- a
message base class, custom messages would inherit from this, and you
just send them out -- lower-level functions handled the rest. The
receive
side would receive the base class, if the message id was a match, then
cast it into the appropriate message class, and utilize the contents at
will.

I can understand how class information is not applicable outside of
certain scopes, but doesn't a struct have the same information
somewhere?
And most importantly, any ideas why this works everywhere else? Being
that it works on three other platforms, I am guessing it is more than
luck.

Any virtual methods in your message base class? The reason I ask is that
your original description of the problem made me think of an object that
contains a virtual table (VTABLE). Different compilers might implement the
VTABLE differently, and it is at least conceivable that your other three
compilers locate them outside the objects. If that is the case, then you
indeed have some reimplementing to do.
When we started looking into the details of the problem, we printed out the
messages in hex, and tried to find all of the class contents that we knew
about for a couple of the shorter messages. We decoded everything except
for the first two bytes on the Win32/Linux systems. On the QNX system, we
noticed the same effect, but the unknown bytes were towards the end of the
message, actually it appeared to be between the base message, and the
inherited message data elements.
->>>--Dale-Sherwood--
New York Air Brake Corp.
TDS Group

Dale Sherwood

Re: Problem sending classes outside of QNX

Post by Dale Sherwood » Wed Aug 06, 2003 7:48 pm

Kevin:

"Kevin White" <kevin.w.white@lmco.com> wrote in message
news:bgrgju$r2q$1@inn.qnx.com...
When we started looking into the details of the problem, we printed out
the
messages in hex, and tried to find all of the class contents that we knew
about for a couple of the shorter messages. We decoded everything except
for the first two bytes on the Win32/Linux systems. On the QNX system, we
noticed the same effect, but the unknown bytes were towards the end of the
message, actually it appeared to be between the base message, and the
inherited message data elements.
At the risk of sounding redundant, the extra bytes look like a virtual table
(resulting from the virtual methods in your base class). Your choices seem
to be:

(1) Eliminate the virtual methods.

(2) Do one of the things that Mario suggested: (a) serialize, or (b) embed a
data structure. A structure embedded in your concrete message classes will
"only" suffer alignment and endian problems, but no virtual table problems.
(Note that you also want to avoid compiler-dependent conveniences like enums
and bitfields.) Sending binary values results in smaller (and faster)
objects, but a text format is more portable (but requires more work to code
since each concrete message class must know how to serialize and unserialize
itself).

->>>--Dale-Sherwood-->
New York Air Brake Corp.
TDS Group

Post Reply

Return to “qdn.public.neutrino”