OpenQNX :: The QNX Community Portal

hello,

not exactly a ddk question, but close enough.

i'm trying to write a resource manager for a device (say /dev/foo) and
i'd like it to support both blocking and non-blocking i/o.

blocking i/o means that a read() from the /dev/foo should block unless
/dev/foo has data to return, or, /dev/foo was open()ed with o_nonblock.

consider the following meta code:

int
my_io_read(resmgr_context_t *ctp, io_read_t *msg, RESMGR_OCB_T *ocb)
{
int status, nonblock;

status = io_read_verify(ctp, msg, ocb, &nonblock);
if (status != EOK)
return (status);

if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
return (ENOSYS);


...

if (!have_data_for_client) {
if (nonblock)
return (EWOULDBLOCK);

/* XXX block until we have data to return */
}

...
}

now, how could i block in my_io_read()?

obviously, my first idea to simply put pthread_conf_wait() to block
calling thread. however, after re-reading available resource manager
documentation, i realized that io_read (and other io_xxx callbacks)
apparently called with iofunc_attr_t locked. without knowing more about
resource manager internals, i assume that i'm allowed to call
iofunc_attr_unlock() in my_io_read(), right?

while searching for the answer, i found out that one can remember
sender's id etc. from io_read msg/ocb and return _RESMGR_NOREPLY from
io_read. later, when data is available, one can send a reply using
previously stored information. is this the only solution to the problem?

thanks,
max

You need to store away the information of the client you wish to block,
and then return _RESMGR_NOREPLY

Later, when you can satify the request, call resmgr_msg_again() and then
do a MsgReplyv to the client with the information.

not exactly a ddk question, but close enough.

i'm trying to write a resource manager for a device (say /dev/foo) and
i'd like it to support both blocking and non-blocking i/o.

blocking i/o means that a read() from the /dev/foo should block unless
/dev/foo has data to return, or, /dev/foo was open()ed with o_nonblock.

consider the following meta code:

int
my_io_read(resmgr_context_t *ctp, io_read_t *msg, RESMGR_OCB_T *ocb)
{
int status, nonblock;

status = io_read_verify(ctp, msg, ocb, &nonblock);
if (status != EOK)
return (status);

if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
return (ENOSYS);


...

if (!have_data_for_client) {
if (nonblock)
return (EWOULDBLOCK);
/* XXX block until we have data to return */
}

...
}

now, how could i block in my_io_read()?

obviously, my first idea to simply put pthread_conf_wait() to block
calling thread. however, after re-reading available resource manager
documentation, i realized that io_read (and other io_xxx callbacks)
apparently called with iofunc_attr_t locked. without knowing more
about resource manager internals, i assume that i'm allowed to call
iofunc_attr_unlock() in my_io_read(), right?

while searching for the answer, i found out that one can remember
sender's id etc. from io_read msg/ocb and return _RESMGR_NOREPLY from
io_read. later, when data is available, one can send a reply using
previously stored information. is this the only solution to the problem?

Colin,

thanks for the quick reply. two questions:

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read before
returning _RESMGR_NOREPLY?

Later, when you can satify the request, call resmgr_msg_again() and
then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more specifically
where do i get a resmgr_context_t pointer? should it be stored (or deep
copied) in my io_read?

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read before
returning _RESMGR_NOREPLY?

Not in terms of the resmgr support, no.

Later, when you can satify the request, call resmgr_msg_again() and
then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more
specifically where do i get a resmgr_context_t pointer? should it be
stored (or deep copied) in my io_read?

resmgr_msg_again is not documented, no. :-(

It does a MsgInfo into ctp->info
then it does a MsgRead of the rcvid into ctp->msg
and then it does a MsgCurrent() (another undocumented kernel call - it
bumps your priority
to that of the client who's waiting)

Take a look at services/slogger for an example of using it.

I wanted to say that your first idea can work, and to me it is
preferable. You can wait in io_read(), typically with a cond_var,
waiting for more data to arrive. If only thread will open the
device at a time, you don't have to worry about the attr being
locked. If you might have multiple readers, you will need to
unlock the attr structure with iofunc_attr_unlock() before you start
waiting, and relock it when awakened.

This all depends on having a multi-threaded resource manager using a
thread pool. This includes making sure that there are an adequate
number of threads for your application, as well as resources to
create those threads.
This model fits much better if you are porting to and from a
Unix/Linux like kernel, and alleviates your need to save and restore
state.

Maksim Yevmenkin wrote:
Colin,

thanks for the quick reply. two questions:

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read before
returning _RESMGR_NOREPLY?

Not in terms of the resmgr support, no.

Later, when you can satify the request, call resmgr_msg_again() and
then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more
specifically where do i get a resmgr_context_t pointer? should it be
stored (or deep copied) in my io_read?

resmgr_msg_again is not documented, no. :-(

It does a MsgInfo into ctp->info
then it does a MsgRead of the rcvid into ctp->msg
and then it does a MsgCurrent() (another undocumented kernel call - it
bumps your priority
to that of the client who's waiting)

Take a look at services/slogger for an example of using it.

Cheers,

Colin

Is there any more information available on resmgr_msg_again and

Colin Burgess wrote:
Maksim Yevmenkin wrote:
Colin,

thanks for the quick reply. two questions:

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read
before returning _RESMGR_NOREPLY?

Not in terms of the resmgr support, no.

Later, when you can satify the request, call resmgr_msg_again() and
then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more
specifically where do i get a resmgr_context_t pointer? should it be
stored (or deep copied) in my io_read?

resmgr_msg_again is not documented, no. :-(

It does a MsgInfo into ctp->info
then it does a MsgRead of the rcvid into ctp->msg
and then it does a MsgCurrent() (another undocumented kernel call - it
bumps your priority
to that of the client who's waiting)

Take a look at services/slogger for an example of using it.

Cheers,

Colin

Is there any more information available on resmgr_msg_again and
MsgCurrent? Perhaps a white paper on when and why they should be used,
or notes on the problems they solve? We make extensive use of resource
managers, and there seems to be something new going on here that we'd
like to be sure we understand.

Murf

funny you should mention this. if fact, the semantic of my resource
manager is that it demands that only one process can access device at a
time.

in fact, i keep track of each open()/close() with my io_open() and
io_close(). i save process id from ctp->info.pid on first open() and
clear it on last close(). later, i check ctp->info.pid in my io_read()
and io_write() against saved one, and if it does not match i return EBUSY.

as it turns out i, can not return EBUSY from io_open(), otherwise i get
nasty errors "device is busy" when i do 'ls /dev/foo' while another
process has it open.

Here's the source to resmgr_msg_again()

int resmgr_msg_again(resmgr_context_t *ctp, int rcvid) {
if(MsgInfo(rcvid, &ctp->info) == -1) {
return -1;
}

if((ctp->info.msglen = MsgRead(ctp->rcvid = rcvid, ctp->msg,
ctp->msg_max_size, 0)) == -1) {
return -1;
}

(void)MsgCurrent(rcvid);

_resmgr_handler(ctp);

return 0;
}

As you can see, it basically refreshes the resmgr_context_t to be the
same as when you originally received the message.
In normal cases you wouldn't need this done.

MsgCurrent() simply adjusts your server threads' priority to that of the
blocked client. I've written docs for it that will be in
a future docs update - I'll post the copy when I dig it out.

Cheers,

Colin

John A. Murphy wrote:
Colin Burgess wrote:
Maksim Yevmenkin wrote:
Colin,

thanks for the quick reply. two questions:

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read
before returning _RESMGR_NOREPLY?

Not in terms of the resmgr support, no.

Later, when you can satify the request, call resmgr_msg_again() and
then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more
specifically where do i get a resmgr_context_t pointer? should it be
stored (or deep copied) in my io_read?

resmgr_msg_again is not documented, no. :-(

It does a MsgInfo into ctp->info
then it does a MsgRead of the rcvid into ctp->msg
and then it does a MsgCurrent() (another undocumented kernel call -
it bumps your priority
to that of the client who's waiting)

Take a look at services/slogger for an example of using it.

Cheers,

Colin

Is there any more information available on resmgr_msg_again and
MsgCurrent? Perhaps a white paper on when and why they should be
used, or notes on the problems they solve? We make extensive use of
resource managers, and there seems to be something new going on here
that we'd like to be sure we understand.

Murf

Yeah, I'd looked at the source to resmgr_msg_again()(and MsgCurrent()),
but it wasn't clear what problem was being solved by the addition of
this code. We'll be anxiously awaiting the new docs....

Murf

Colin Burgess wrote:
Here's the source to resmgr_msg_again()

int resmgr_msg_again(resmgr_context_t *ctp, int rcvid) {
if(MsgInfo(rcvid, &ctp->info) == -1) {
return -1;
}

if((ctp->info.msglen = MsgRead(ctp->rcvid = rcvid, ctp->msg,
ctp->msg_max_size, 0)) == -1) {
return -1;
}

(void)MsgCurrent(rcvid);

_resmgr_handler(ctp);

return 0;
}

As you can see, it basically refreshes the resmgr_context_t to be the
same as when you originally received the message.
In normal cases you wouldn't need this done.

MsgCurrent() simply adjusts your server threads' priority to that of
the blocked client. I've written docs for it that will be in
a future docs update - I'll post the copy when I dig it out.

Cheers,

Colin

John A. Murphy wrote:
Colin Burgess wrote:
Maksim Yevmenkin wrote:
Colin,

thanks for the quick reply. two questions:

You need to store away the information of the client you wish to
block, and then return _RESMGR_NOREPLY

1) do i need to store anything else besides rcvid in my io_read
before returning _RESMGR_NOREPLY?

Not in terms of the resmgr support, no.

Later, when you can satify the request, call resmgr_msg_again()
and then do a MsgReplyv to the client with the information.

2) where can i read more about resmgr_msg_again() and more
specifically where do i get a resmgr_context_t pointer? should it
be stored (or deep copied) in my io_read?

resmgr_msg_again is not documented, no. :-(

It does a MsgInfo into ctp->info
then it does a MsgRead of the rcvid into ctp->msg
and then it does a MsgCurrent() (another undocumented kernel call -
it bumps your priority
to that of the client who's waiting)

Take a look at services/slogger for an example of using it.

Cheers,

Colin

Is there any more information available on resmgr_msg_again and
MsgCurrent? Perhaps a white paper on when and why they should be
used, or notes on the problems they solve? We make extensive use of
resource managers, and there seems to be something new going on here
that we'd like to be sure we understand.

Murf

Maksim Yevmenkinwrote:
funny you should mention this. if fact, the semantic of my resource
manager is that it demands that only one process can access device
at a
time.

in fact, i keep track of each open()/close() with my io_open() and
io_close(). i save process id from ctp->info.pid on first open()
and
clear it on last close(). later, i check ctp->info.pid in my
io_read()
and io_write() against saved one, and if it does not match i return
EBUSY.
as it turns out i, can not return EBUSY from io_open(), otherwise i
get
nasty errors "device is busy" when i do 'ls /dev/foo'
while another
process has it open.


I think there is a better way to handle this. You can distinquish
between an open caused by ls from a user trying to get access. You
could then process the former, but EBUSY the latter.