usrp-users@lists.ettus.com

Discussion and technical support related to USRP, UHD, RFNoC

View all threads

RFNoC CHDR DstEPID and Virtual Channels Clarification

BP
Brian Padalino
Fri, Apr 18, 2025 4:37 AM

I have a design for an X440 that wants to have 128 virtual channels for
each of the ports of a 4-port block.

The VC field in the header is only 6 bits, but the DstEPID is 16 bits.

Is there some way to let the endpoint number be shifted up in the DstEPID
and pass through the bottom 7 bits? I'm using the chdr_to_axis_pyld_ctxt to
separate it out, but the DstEPID always seems to be 0 at least in my
testbench. It seems like maybe the EPID is dynamically assigned during some
type of configuration stage?

I guess the other part of this is how I might be able to craft a CHDR
packet on the UHD side of things to send it down to my block.

Can a block have multiple DstEPID's assigned to it?

If there is another way I could get the 128 virtual channels per RFNoC
block port, please let me know. I am very much open to suggestions.

Thanks,
Brian

I have a design for an X440 that wants to have 128 virtual channels for each of the ports of a 4-port block. The VC field in the header is only 6 bits, but the DstEPID is 16 bits. Is there some way to let the endpoint number be shifted up in the DstEPID and pass through the bottom 7 bits? I'm using the chdr_to_axis_pyld_ctxt to separate it out, but the DstEPID always seems to be 0 at least in my testbench. It seems like maybe the EPID is dynamically assigned during some type of configuration stage? I guess the other part of this is how I might be able to craft a CHDR packet on the UHD side of things to send it down to my block. Can a block have multiple DstEPID's assigned to it? If there is another way I could get the 128 virtual channels per RFNoC block port, please let me know. I am very much open to suggestions. Thanks, Brian
WF
Wade Fife
Mon, Apr 28, 2025 12:43 AM

The EPIDs are only used by the stream endpoints. I think it will always be
set to 0 when it is sent to your block, and anything you put in there will
be ignored by downstream blocks. In theory, I suppose it could be used the
way you described, but I think that would require pretty significant
changes in both UHD and in the FPGA.

Could you use two stream endpoints instead to double the number of virtual
channels?

Thanks,

Wade

On Thu, Apr 17, 2025 at 11:38 PM Brian Padalino bpadalino@gmail.com wrote:

I have a design for an X440 that wants to have 128 virtual channels for
each of the ports of a 4-port block.

The VC field in the header is only 6 bits, but the DstEPID is 16 bits.

Is there some way to let the endpoint number be shifted up in the DstEPID
and pass through the bottom 7 bits? I'm using the chdr_to_axis_pyld_ctxt to
separate it out, but the DstEPID always seems to be 0 at least in my
testbench. It seems like maybe the EPID is dynamically assigned during some
type of configuration stage?

I guess the other part of this is how I might be able to craft a CHDR
packet on the UHD side of things to send it down to my block.

Can a block have multiple DstEPID's assigned to it?

If there is another way I could get the 128 virtual channels per RFNoC
block port, please let me know. I am very much open to suggestions.

Thanks,
Brian


USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com

The EPIDs are only used by the stream endpoints. I think it will always be set to 0 when it is sent to your block, and anything you put in there will be ignored by downstream blocks. In theory, I suppose it could be used the way you described, but I think that would require pretty significant changes in both UHD and in the FPGA. Could you use two stream endpoints instead to double the number of virtual channels? Thanks, Wade On Thu, Apr 17, 2025 at 11:38 PM Brian Padalino <bpadalino@gmail.com> wrote: > I have a design for an X440 that wants to have 128 virtual channels for > each of the ports of a 4-port block. > > The VC field in the header is only 6 bits, but the DstEPID is 16 bits. > > Is there some way to let the endpoint number be shifted up in the DstEPID > and pass through the bottom 7 bits? I'm using the chdr_to_axis_pyld_ctxt to > separate it out, but the DstEPID always seems to be 0 at least in my > testbench. It seems like maybe the EPID is dynamically assigned during some > type of configuration stage? > > I guess the other part of this is how I might be able to craft a CHDR > packet on the UHD side of things to send it down to my block. > > Can a block have multiple DstEPID's assigned to it? > > If there is another way I could get the 128 virtual channels per RFNoC > block port, please let me know. I am very much open to suggestions. > > Thanks, > Brian > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >
BP
Brian Padalino
Mon, Apr 28, 2025 1:27 AM

On Sun, Apr 27, 2025 at 8:44 PM Wade Fife wade.fife@ettus.com wrote:

The EPIDs are only used by the stream endpoints. I think it will always be
set to 0 when it is sent to your block, and anything you put in there will
be ignored by downstream blocks. In theory, I suppose it could be used the
way you described, but I think that would require pretty significant
changes in both UHD and in the FPGA.

Could you use two stream endpoints instead to double the number of virtual
channels?

Maybe I could do that. So have 4 endpoints be the lower 64 channels, and
then have 4 endpoints be the upper 64 channels and utilize only the VC
field?

When it comes to SeqNum, do the VC's count as being separate or would the
SeqNum also increase since the DestEPID is still the same?

Thanks,
Brian

On Sun, Apr 27, 2025 at 8:44 PM Wade Fife <wade.fife@ettus.com> wrote: > The EPIDs are only used by the stream endpoints. I think it will always be > set to 0 when it is sent to your block, and anything you put in there will > be ignored by downstream blocks. In theory, I suppose it could be used the > way you described, but I think that would require pretty significant > changes in both UHD and in the FPGA. > > Could you use two stream endpoints instead to double the number of virtual > channels? > Maybe I could do that. So have 4 endpoints be the lower 64 channels, and then have 4 endpoints be the upper 64 channels and utilize only the VC field? When it comes to SeqNum, do the VC's count as being separate or would the SeqNum also increase since the DestEPID is still the same? Thanks, Brian
WF
Wade Fife
Mon, Apr 28, 2025 12:46 PM

Right, up to 64 VCs per stream endpoint.

Each stream endpoint only supports a single sequence number counter, so
that single counter would be shared by all virtual channels.

Wade

On Sun, Apr 27, 2025 at 8:27 PM Brian Padalino bpadalino@gmail.com wrote:

On Sun, Apr 27, 2025 at 8:44 PM Wade Fife wade.fife@ettus.com wrote:

The EPIDs are only used by the stream endpoints. I think it will always
be set to 0 when it is sent to your block, and anything you put in there
will be ignored by downstream blocks. In theory, I suppose it could be used
the way you described, but I think that would require pretty significant
changes in both UHD and in the FPGA.

Could you use two stream endpoints instead to double the number of
virtual channels?

Maybe I could do that. So have 4 endpoints be the lower 64 channels, and
then have 4 endpoints be the upper 64 channels and utilize only the VC
field?

When it comes to SeqNum, do the VC's count as being separate or would the
SeqNum also increase since the DestEPID is still the same?

Thanks,
Brian

Right, up to 64 VCs per stream endpoint. Each stream endpoint only supports a single sequence number counter, so that single counter would be shared by all virtual channels. Wade On Sun, Apr 27, 2025 at 8:27 PM Brian Padalino <bpadalino@gmail.com> wrote: > On Sun, Apr 27, 2025 at 8:44 PM Wade Fife <wade.fife@ettus.com> wrote: > >> The EPIDs are only used by the stream endpoints. I think it will always >> be set to 0 when it is sent to your block, and anything you put in there >> will be ignored by downstream blocks. In theory, I suppose it could be used >> the way you described, but I think that would require pretty significant >> changes in both UHD and in the FPGA. >> >> Could you use two stream endpoints instead to double the number of >> virtual channels? >> > > Maybe I could do that. So have 4 endpoints be the lower 64 channels, and > then have 4 endpoints be the upper 64 channels and utilize only the VC > field? > > When it comes to SeqNum, do the VC's count as being separate or would the > SeqNum also increase since the DestEPID is still the same? > > Thanks, > Brian >
BP
Brian Padalino
Mon, Apr 28, 2025 12:56 PM

On Mon, Apr 28, 2025 at 8:47 AM Wade Fife wade.fife@ettus.com wrote:

Right, up to 64 VCs per stream endpoint.

Each stream endpoint only supports a single sequence number counter, so
that single counter would be shared by all virtual channels.

Thanks for the clarification on this.

From your perspective, what is the feasibility of having 1024 individual
endpoints in a system?

Thanks,
Brian

On Mon, Apr 28, 2025 at 8:47 AM Wade Fife <wade.fife@ettus.com> wrote: > Right, up to 64 VCs per stream endpoint. > > Each stream endpoint only supports a single sequence number counter, so > that single counter would be shared by all virtual channels. > Thanks for the clarification on this. From your perspective, what is the feasibility of having 1024 individual endpoints in a system? Thanks, Brian >
WF
Wade Fife
Mon, Apr 28, 2025 2:33 PM

In practice, you can't have a large number of stream endpoints in a single
USRP, because the crossbar and associated logic adds up. Something on the
order of 16 or so might be a practical limit, depending a lot on what's on
those endpoints. If you need to distinguish between more data streams, then
you'd want to use something like virtual channels or prepending your data
with some kind of identifier.

With many USRPs interconnected, you could have a lot more endpoints in
theory.

Wade

On Mon, Apr 28, 2025 at 7:56 AM Brian Padalino bpadalino@gmail.com wrote:

On Mon, Apr 28, 2025 at 8:47 AM Wade Fife wade.fife@ettus.com wrote:

Right, up to 64 VCs per stream endpoint.

Each stream endpoint only supports a single sequence number counter, so
that single counter would be shared by all virtual channels.

Thanks for the clarification on this.

From your perspective, what is the feasibility of having 1024 individual
endpoints in a system?

Thanks,
Brian

In practice, you can't have a large number of stream endpoints in a single USRP, because the crossbar and associated logic adds up. Something on the order of 16 or so might be a practical limit, depending a lot on what's on those endpoints. If you need to distinguish between more data streams, then you'd want to use something like virtual channels or prepending your data with some kind of identifier. With many USRPs interconnected, you could have a lot more endpoints in theory. Wade On Mon, Apr 28, 2025 at 7:56 AM Brian Padalino <bpadalino@gmail.com> wrote: > On Mon, Apr 28, 2025 at 8:47 AM Wade Fife <wade.fife@ettus.com> wrote: > >> Right, up to 64 VCs per stream endpoint. >> >> Each stream endpoint only supports a single sequence number counter, so >> that single counter would be shared by all virtual channels. >> > > Thanks for the clarification on this. > > From your perspective, what is the feasibility of having 1024 individual > endpoints in a system? > > Thanks, > Brian > >>
BP
Brian Padalino
Mon, Apr 28, 2025 2:37 PM

On Mon, Apr 28, 2025 at 10:33 AM Wade Fife wade.fife@ettus.com wrote:

In practice, you can't have a large number of stream endpoints in a single
USRP, because the crossbar and associated logic adds up. Something on the
order of 16 or so might be a practical limit, depending a lot on what's on
those endpoints. If you need to distinguish between more data streams, then
you'd want to use something like virtual channels or prepending your data
with some kind of identifier.

Thanks for this answer.

Does static routing help with this or not particularly?

I have only a single configuration I ever want to run, and it's endpoints
directly into a modified radio block - no other blocks.

Thanks,
Brian

On Mon, Apr 28, 2025 at 10:33 AM Wade Fife <wade.fife@ettus.com> wrote: > In practice, you can't have a large number of stream endpoints in a single > USRP, because the crossbar and associated logic adds up. Something on the > order of 16 or so might be a practical limit, depending a lot on what's on > those endpoints. If you need to distinguish between more data streams, then > you'd want to use something like virtual channels or prepending your data > with some kind of identifier. > Thanks for this answer. Does static routing help with this or not particularly? I have only a single configuration I ever want to run, and it's endpoints directly into a modified radio block - no other blocks. Thanks, Brian >
WF
Wade Fife
Thu, May 1, 2025 2:40 AM

I don't think static routing will help in your case. Stream endpoints are
only for communicating over dynamic routes and the EPID is used for that
routing.

Wade

On Mon, Apr 28, 2025 at 9:37 AM Brian Padalino bpadalino@gmail.com wrote:

On Mon, Apr 28, 2025 at 10:33 AM Wade Fife wade.fife@ettus.com wrote:

In practice, you can't have a large number of stream endpoints in a
single USRP, because the crossbar and associated logic adds up. Something
on the order of 16 or so might be a practical limit, depending a lot on
what's on those endpoints. If you need to distinguish between more data
streams, then you'd want to use something like virtual channels or
prepending your data with some kind of identifier.

Thanks for this answer.

Does static routing help with this or not particularly?

I have only a single configuration I ever want to run, and it's endpoints
directly into a modified radio block - no other blocks.

Thanks,
Brian

I don't think static routing will help in your case. Stream endpoints are only for communicating over dynamic routes and the EPID is used for that routing. Wade On Mon, Apr 28, 2025 at 9:37 AM Brian Padalino <bpadalino@gmail.com> wrote: > On Mon, Apr 28, 2025 at 10:33 AM Wade Fife <wade.fife@ettus.com> wrote: > >> In practice, you can't have a large number of stream endpoints in a >> single USRP, because the crossbar and associated logic adds up. Something >> on the order of 16 or so might be a practical limit, depending a lot on >> what's on those endpoints. If you need to distinguish between more data >> streams, then you'd want to use something like virtual channels or >> prepending your data with some kind of identifier. >> > > Thanks for this answer. > > Does static routing help with this or not particularly? > > I have only a single configuration I ever want to run, and it's endpoints > directly into a modified radio block - no other blocks. > > Thanks, > Brian > >>
MB
Martin Braun
Tue, May 6, 2025 7:58 AM

One thing to keep in mind: If you have many many virtual channels, then you
probably still have a lot of blocks. The control crossbar also increases
with N^2 in size, where N is the number of blocks (not the number of stream
endpoints, which affects the size of the CHDR crossbar).

Maybe what you're trying to do is better handled at the application level.
Meaning that you have bigger blocks that do some kind of channelization
internally, and then pull apart the stream in your application. After all,
there are very few rules on what kind of data you can stream to the host
(or vice versa). For example, when we stream FFT data to the host, we don't
separate the bins into virtual channels, but we put the onus on the
application to understand that every vector of FFT-size needs to be
understood as such.

Not sure if this is helpful to you -- of course, for most cases, RFNoC
should be doing all the hard work for you. But in some cases, it might be
easier to do some of the work in your app.

--M

On Thu, May 1, 2025 at 4:42 AM Wade Fife wade.fife@ettus.com wrote:

I don't think static routing will help in your case. Stream endpoints are
only for communicating over dynamic routes and the EPID is used for that
routing.

Wade

On Mon, Apr 28, 2025 at 9:37 AM Brian Padalino bpadalino@gmail.com
wrote:

On Mon, Apr 28, 2025 at 10:33 AM Wade Fife wade.fife@ettus.com wrote:

In practice, you can't have a large number of stream endpoints in a
single USRP, because the crossbar and associated logic adds up. Something
on the order of 16 or so might be a practical limit, depending a lot on
what's on those endpoints. If you need to distinguish between more data
streams, then you'd want to use something like virtual channels or
prepending your data with some kind of identifier.

Thanks for this answer.

Does static routing help with this or not particularly?

I have only a single configuration I ever want to run, and it's endpoints
directly into a modified radio block - no other blocks.

Thanks,
Brian


USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com

One thing to keep in mind: If you have many many virtual channels, then you probably still have a lot of blocks. The control crossbar also increases with N^2 in size, where N is the number of blocks (not the number of stream endpoints, which affects the size of the CHDR crossbar). Maybe what you're trying to do is better handled at the application level. Meaning that you have bigger blocks that do some kind of channelization internally, and then pull apart the stream in your application. After all, there are very few rules on what kind of data you can stream to the host (or vice versa). For example, when we stream FFT data to the host, we don't separate the bins into virtual channels, but we put the onus on the application to understand that every vector of FFT-size needs to be understood as such. Not sure if this is helpful to you -- of course, for most cases, RFNoC should be doing all the hard work for you. But in some cases, it might be easier to do some of the work in your app. --M On Thu, May 1, 2025 at 4:42 AM Wade Fife <wade.fife@ettus.com> wrote: > I don't think static routing will help in your case. Stream endpoints are > only for communicating over dynamic routes and the EPID is used for that > routing. > > Wade > > > On Mon, Apr 28, 2025 at 9:37 AM Brian Padalino <bpadalino@gmail.com> > wrote: > >> On Mon, Apr 28, 2025 at 10:33 AM Wade Fife <wade.fife@ettus.com> wrote: >> >>> In practice, you can't have a large number of stream endpoints in a >>> single USRP, because the crossbar and associated logic adds up. Something >>> on the order of 16 or so might be a practical limit, depending a lot on >>> what's on those endpoints. If you need to distinguish between more data >>> streams, then you'd want to use something like virtual channels or >>> prepending your data with some kind of identifier. >>> >> >> Thanks for this answer. >> >> Does static routing help with this or not particularly? >> >> I have only a single configuration I ever want to run, and it's endpoints >> directly into a modified radio block - no other blocks. >> >> Thanks, >> Brian >> >>> _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >