Cywee Group LTD v. Apple Inc.
Filing
1
COMPLAINT for Patent Infringement against Apple Inc. ( Filing fee $ 400, receipt number 0971-8556570.). Filed byCywee Group LTD. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Civil Cover Sheet)(Kopeikin, Jill) (Filed on 4/22/2014)
EXHIBIT B
US00844143 8B2
(12) United States Patent
Ye et al.
(54)
(10) Patent No.:
US 8,441,438 B2
(45) Date of Patent:
May 14, 2013
3D POINTING DEVICE AND METHOD FOR
7,414,611 B2
8/2008 Liberty
COMPENSATING MOVEMENT THEREOF
7,489,298 B2
7,535,456 B2
2/2009 hbe?y er a1~
5/2009 Liberty et a1.
7,774,155 B2*
8/2010
.
(75) Inventors: Zhf’“ Ye’ F051“ C113” CA (Us);
345/158
7,924,264 132*
8,010,313 B2 *
ChlIl-Lllllg L1, Taoyuan County (TW);
Shun-Nan Liou, Kaohsiung (TW)
345/157
702/141
2008/0096654 A1*
'
.
-
-
(73)
Asslgnee- CyWee GFOIIP Llmlted, TOITOla (VG)
(*)
Notice:
Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U_S_C_ 154(1)) by 218 days_
Sato et a1. ................... .. 702/127
7,817,134 B2* 10/2010 Huang et a1.
2009/0262074 A1*
4/2011 Ohta ............ ..
8/2011 Mathews et a1.
4/2008 Mondesir et a1. ..
10/2009
Nasiri et a1.
..... ..
463/31
345/158
2011/0307173 A1,, 12/2011 Riley ““““““““““““““ “ 701/220
OTHER PUBLICATIONS
AZuma, Ronald et a1. Improving Static and Dynamic Registration in
an Optical See-Through HMD. Proceedings of SIGGRAPH ’94
21
APP 1. No.: 12/943 s 934
(Orlando, Fla., Jul. 24 29, 1994), Computer Graphics, Annual Con
ference Series, 1994, 197 204*
(22) Filed:
Nov. 11, 2010
* cited by examiner
(65)
Prior Publication Data
Us 2011/0163950 A1
Jul 7 2011
Primary Examiner * William Boddie
’
Assistant Examiner * Bryan E Earles
Related US. Application Data
(74) Attorney, Agent, or Firm * Ding Yu Tan
(60) l6’ro2\(/)i1s(i)onal application No. 61/292,558, ?led on Jan.
’
i
G09G 5/00
us CL
ABSTRACT
A three-dimensional (3D) pointing device capable of accu
(51) Int, Cl,
(52)
(57)
rately outputting a deviation including yaW, pitch and roll
(200601)
angles in a 3D reference frame and preferably in an absolute
manner is provided. Said 3D pointing device comprises a
USPC ........................................................ .. 345/156
six'axis motion Sensor module including a rotation Sensor
Field of Classi?cation Search ...................... .. None
andanaccelerometen andaprocessing andtransmimng mod‘
See application ?le for Complete Search history
(58)
ule. The six-axis motion sensor module generates a ?rst sig
nal set comprising angular velocities and a second signal set
comprising axial accelerations associated With said move
ments and rotations of the 3D pointing device in the 3D
(56)
References Cited
reference frame. The processing and transmitting module
U.S. PATENT DOCUMENTS
5,138,154 A
8/1992 Hotelling
5,440,326 A
8/1995 Quinn
5,898,421 A
4/1999 Quinn
utilizes a comparison method to compare the ?rst signal set
With the second signal set to obtain an updated state of the
6,993,451 B2*
6,061,611 A *
measured state thereof in order to output the resulting devia
tion in the 3D reference frame and preferably in an absolute
six-axis motion sensor module based on a current state and a
5/2000 Whitmorea1.
1/2006 Chang et
7,158,118 B2
1/2007 Liberty
7,236,156 B2
7,239,301 B2
6/2007 Liberty et a1.
7/2007 Liberty et a1.
7,262,760 B2
8/2007 Liberty
manner.
19 Claims, 7 Drawing Sheets
705
Initialize an initial-value
set
710
Obtain a previous state
715
(1st quatemion) at T-l
Obtain measured angular
velocities at T
720
Obtain a current state
(2nd quatemian) at ‘l
Obtain ‘measured axial
725
accelerations‘ of a
measured state at T
730
CGICUMG 'predicted axial
accelerations’ of a
measured state at T
Output 3rd quaternion
to is! quatemion
Obtain resultant
deviation including yaw,
pitch and roll angles
Obtain display data and
translate the resultant
angles to movement
pattern in the display
reference frame
Obtain an updated state
735
(3rd quaternion) by
comparing current state
with measured state
US. Patent
May 14, 2013
Sheet 1 017
2
US 8,441,438 B2
D
120
X0
122
Yo
XP
110
113
YP
112
111
ZP
FIG. 1 (RELATED ART)
Zn
120
X0
122
Y
YP
D
112
110
XP
113
ZP
111
FIG. 2 (RELATED ART)
US. Patent
May 14, 2013
Sheet 2 on
US 8,441,438 B2
310\
320)
FIG. 3
US. Patent
May 14, 2013
Sheet 3 of7
US 8,441,438 B2
Rotation
Data
Transmitting
Unit
FIG 0
4
Computing
Processor
US. Patent
May 14, 2013
Sheet 4 on
US 8,441,438 B2
620“
FIG. 6
US. Patent
May 14, 2013
Sheet 5 0f 7
r
7051
US 8,441,438 B2
\
lnitialize an initial-value
set
a
J
Obtain a previous state
7101
(1st quaternion) at T-l
L
J
F
\
Obtain measured
7151 velocities at T angular
720 1 Obtain a current state
Output 3rd quaternion
(2nd quaternion) at T
to 1st quaternion
740
,
Obtain "measured axial
Obtain resultant
measured state at T
deviation including yaw,
pitch and roll angles
accelerations of a
7251
L
J
W
‘\
Calculate "predicted axial
accelerations of a
measured state at T
Obtain an updated state
7351/ (3rd quaternion) by
comparing current state
with measured state
FIG. 7
745
US. Patent
705
May 14, 2013
Sheet 6 of7
US 8,441,438 B2
Initialize an initial-value
set
710
Obtain a previous state
(1st quaternion) at T-l J
f
\
715x Obtain measured angular
velocities at T
k
J
720x Obtain a current state
.
~
t(2nd quatermon) at T a
Output 3rd quaternion
to lst quaternion
1740
__—__J
fObtain "measured axial
Obtain resultant
725x accelerations” of a
measured state at T
O
J
730x Calculate "predicted axial
accelerations” of a
measured state at T
deviation including yaw, $745
pitch and roll angles
%
Obtain display data and
translate the resultant
k
j
angles to movement I750
pattern in the display
reference frame
Obtain an updated state
735x (3rd quaternion) by
comparing current state
with measured state
\
J
FIG. 8
US. Patent
May 14, 2013
Sheet 7 0f 7
Pmox
FIG. 9
US 8,441,438 B2
US 8,441,438 B2
1
2
3D POINTING DEVICE AND METHOD FOR
COMPENSATING MOVEMENT THEREOF
device 110 about theYP axis; the roll angle 113 may represent
the rotation of the pointing device 110 about the XP axis.
In a knoWn related art as shoWn in FIG. 1, When the yaW
angle 111 of the pointing device 110 changes, the aforemen
CROSS-REFERENCE TO RELATED
APPLICATION
tioned pointer on the screen 122 must move horizontally or in
a horizontal direction With reference to the ground in
response to the change of the yaW angle 111. FIG. 2 shoWs
What happens When the user rotates the pointing device 110
counterclockwise by a degree such as a 90-degree about the
This application claims priority bene?ts of Us. Patent
Provisional Application No. 61/292,558, ?led on Jan. 6,
2010. The entirety of the above-mentioned patent applica
tions is hereby incorporated by reference herein and made a
part of this speci?cation.
XP axis.
In another knoWn related art as shoWn in FIG. 2, When the
yaW angle 111 changes, the aforementioned pointer on the
screen 122 is expected to move vertically in response. The
change of the yaW angle 111 can be detected by a gyro-sensor
BACKGROUND OF THE INVENTION
Which detects the angular velocity 00x of the pointing device
1. Field of the Invention
The present invention generally relates to a three-dimen
sional (3D) pointing device utilizing a motion sensor module
110 about the XP axis. FIG. 1 and FIG. 2 shoW that the same
change of the yaW angle 111 may be mapped to different
movements of the point on the screen 122. Therefore, a proper
and method of compensating and mapping signals of the
motion sensor module subject to movements and rotations of
20
compensation mechanism for the orientation of the pointing
device 110 is required such that corresponding mapping of
said 3D pointing device. More particularly, the present inven
the pointer on the screen 122 of the display 120 may be
tion relates to a 3D pointing device utilizing a six-axis motion
sensor module With an enhanced comparison to calculate and
compensate accumulated errors associated With the motion
sensor module and to obtain actual resulting deviation angles
obtained correctly and desirably. The term compensation of
the prior arts by Liberty (US. Pat. No. 7,158,118, U.S. Pat.
No. 7,262,760 and Us. Pat. No. 7,414,611) refers to the
25
in spatial reference frame and under dynamic environments.
2. Description of the Related Art
FIG. 1 is a schematic diagram shoWing a user using a
handheld 3D pointing device 110 to point at a point on the
screen 122 of a 2D display device 120. If the pointing device
effects or extra rotations about the axis related to “roll”. The
30
110 emits a light beam, the point Would be the location Where
the light beam hits the screen 122. For example, the pointing
device 110 may be a mouse of a computer or a pad of a video
game console. The display device 120 may be a part of the
computer or the video game console. There are tWo reference
correction and compensation of signals subject to gravity
35
frames, such as the spatial pointer reference frame and the
display frame, associated With the pointing device 110 and
the display device 120, respectively. The ?rst reference frame
or spatial pointer reference frame associated With the pointing
term of “comparison” of the present invention may generally
refer to the calculating and obtaining of the actual deviation
angles of the 3D pointing device 110 With respect to the ?rst
reference frame or spatial pointing frame XPYPZP utilizing
signals generated by motion sensors While reducing or elimi
nating noises associated With said motion sensors; Whereas
the term mapping may refer to the calculating and translating
of said deviation angles in the sptatial pointing frame
XPYPZP onto the aforementioned pointer on the display
plane associated With the 2D display device 120 of a second
reference frame or display frame XDYDZD.
It is knoWn that a pointing device utilizing 5-axis motion
shoWn in FIG. 1. The second reference frame or display frame
associated With the display device 120 is de?ned by the coor
sensors, namely, Ax, Ay, Az, uuYand 002 may be compensated.
For example, U.S. Pat. No. 7,158,118 by Liberty, U.S. Pat.
No. 7,262,760 by Liberty and Us. Pat. No. 7,414,611 by
Liberty provide such pointing device having a 5-axis motion
dinate axes XD, YD and ZD as shoWn in FIG. 1. The screen 122
sensor and discloses a compensation using tWo gyro-sensors
device 110 is de?ned by the coordinate axes XP, YP and ZP as
ofthe display device 120 is a subset ofthe XDYD plane ofthe
reference frame XDYDZD associated With the display device
120. Therefore, the XDYD plane is also knoWn as the display
plane associated With the display device 120.
40
45
XPYPZP. The pointing device by Liberty utilizing a 5-axis
motion sensor may not output deviation angles of the pointing
device in, for example, a 3D reference frame; in other Words,
A user may perform control actions and movements utiliz
ing the pointing device for certain purposes including enter
tainment such as playing a video game, on the display device
120 through the aforementioned pointer on the screen 122.
50
due to due to the limitation of the 5-axis motion sensor of
accelerometers and gyro-sensors utilized therein, the point
ing device by Liberty cannot output deviation angles readily
For proper interaction With the use of the pointing device,
When the user moves the pointing device 110, the pointer on
the screen 122 is expected to move along With the orientation,
direction and distance travelled by the pointing device 110
and the display 120 shall display such movement of the
my and 002 to detect rotation about the Yp and Zp axes, and
accelerometers Ax, Ay and Az to detect the acceleration of the
pointing device along the three axes of the reference frame
55
in 3D reference frame but rather a 2D reference frame only
and the output of such device having 5-axis motion sensors is
a planar pattern in 2D reference frame only. In addition, it has
been found that the pointing device and compensation dis
pointer to a neW location on the screen 122 of the display 120.
closed therein cannot accurately or properly calculate or
The orientation of the pointing device 1 10 may be represented
by three deviation angles of the 3D pointing device 110 With
respect to the reference frame XPYPZP, namely, the yaW
angle 1 11, the pitch angle 1 12 and the roll angle 1 13. The yaW,
pitch and roll angles 111, 112, 113 may be best understood in
relation to the universal standard de?nition of spatial angles
obtain movements, angles and directions of the pointing
device While being subject to unexpected dynamic movement
60
and/or accelerations along With the direction of gravity. In
other Words, it has been found that dynamic actions or extra
accelerations including additional accelerations, in particular
related to commercial vehicles or transportation such as ships
and airplanes. Conventionally, the yaW angle 111 may repre
sent the rotation of the pointing device 110 about the Z1, axis;
the pitch angle 112 may represent the rotation of the pointing
during the obtaining of the signals generated by the motion
sensors, in particular, during unexpected drifting movements
65
the one acted upon the direction substantially parallel to or
along With the gravity imposed on the pointing device With
the compensation methods provided by Liberty, said pointing
US 8,441,438 B2
3
4
device by Liberty cannot properly or accurately output the
actual yaw, pitch and roll angles in the spatial reference frame
XPYPZP and following which, consequently, the mapping of
frame may too be further mapped or translated to a pattern
useful in a 2D reference frame.
the spatial angles onto any 2D display reference frame such as
SUMMARY OF THE INVENTION
XDYDZD may be greatly affected and erred. To be more
speci?c, as the 5-axis compensation by Liberty cannot detect
According to one aspect of an example embodiment of the
present invention, a 3D pointing device utilizing a six-axis
motion sensor module is provided. The 3D pointing device
or compensate rotation about the XP axis directly or accu
rately, the rotation about the XP axis has to be derived from the
gravitational acceleration detected by the accelerometer. Fur
thermore, the reading of the accelerometer may be accurate
only when the pointing device is static since due to the limi
comprises an accelerometer to measure or detect axial accel
erations Ax, AZ, Ay and a rotation sensor to measure or detect
tation on known accelerometers that these sensors may not
including resultant angles comprising yaw, pitch and roll
distinguish the gravitational acceleration from the accelera
tion of the forces including centrifugal forces or other types of
additional accelerations imposed or exerted by the user.
Furthermore, it has been found that known prior arts may
angles in a spatial pointer frame of the 3D pointing device
subject to movements and rotations in dynamic environments
may be obtained and such that said resulting deviation includ
ing said resultant angles may be obtained and outputted in an
only be able to output a “relative” movement pattern in a 2D
reference frame based on the result calculated from the sig
nals of motion sensors. For example, the abovementioned
absolute manner re?ecting or associating with the actual
movements and rotations of the 3D pointer device of the
angular velocities 00x, my, 002 such that resulting deviation
in a relative manner and a pointer on a display screen to show
present invention in said spatial pointer reference frame.
According to another aspect of the present invention, the
present invention provides an enhanced comparison method
such corresponding 2D relative movement pattern. To be
to eliminate the accumulated errors as well as noises over
more speci?c, the pointer moves from a ?rst location to a
time associated with signals generated by a combination of
motion sensors, including the ones generated by accelerom
eters Ax, Ay, AZ and the ones generated by gyroscopes 00x, my,
002 in dynamic environments. In other words, accumulated
prior arts by Liberty may only output a 2D movement pattern
second new location relative to said ?rst location only. Such
relative movement from the previous location to the next
location with respect to time cannot accurately determine
and/ or output the next location, particularly in situations
where the previous location may have been an erred location
20
25
or have been faultily determined as an incorrect reference
point for the next location that is to be calculated therefrom
and obtained based on their relative relationship adapted. One
30
errors associated with a fusion of signals from a motions
sensor module comprising a plurality of motion sensors to
detect movements on and rotations about different axes of a
reference frame may be eliminated or corrected.
illustration of such defect of known prior arts adapting a
relative relationship in obtaining a movement pattern may be
According to still another aspect of the present invention,
the present invention provides an enhanced comparison
method to correctly calculating and outputting a resulting
clearly illustrated by an example showing the faultily output
deviation comprising a set of resultant angles including yaw,
ted movements of a pointer intended to move out of a bound 35 pitch and roll angles in a spatial pointer frame, preferably
about each of three orthogonal coordinate axes of the spatial
ary or an edge of display screen. It has been found that as the
pointer reference frame, by comparing signals of rotation
pointer of known prior arts reaches the edge of a display and
continues to move out of the boundary or edge at a certain
sensor related to angular velocities or rates with the ones of
extra extent beyond said boundary, the pointer fails to dem
accelerometer related to axial accelerations such that these
onstrate a correct or “absolute” pattern as it moves to a new 40
angles may be accurately outputted and obtained, which may
location either within the display or remaining outside of the
boundary; in other words, instead of returning to a new loca
tion by taking into account said certain extra extend beyond
the boundary made earlier in an “absolute” manner, the
pointer of known arts discards such virtual distance of the
extra extend beyond the boundary already made and an erred
next position is faultily outputted due to the relative relation
ship adapted and utiliZed by the pointer. may be never calcu
lated or processed due to the faultily obtained location at the
edge or boundary of the display as well as the relative rela
tionship adapted to obtain its next location therefrom.
Therefore, it is clear that an improved pointing device with
enhanced calculating or comparison method capable of accu
too be further mapping to another reference frame different
from said spatial pointer frame.
According to still another aspect of the present invention,
45
reference frame onto a display frame such that a movement
pattern in a display frame different from the spatial pointer
50
reference frame may be obtained according to the mapping or
translation of the resultant angles of the resultant deviation
onto said movement pattern.
According to another example embodiment of the present
rately obtaining and calculating actual deviation angles in the
spatial pointer frame as well as mapping of such angles onto
a pointer on the display frame in dynamic environments and
conditions is needed. In addition, as the trend of 3D technol
ogy advances and is applicable to various ?elds including
displays and interactive systems, there is a signi?cant need for
a 3D pointing device capable of accurately outputting a devia
tion of such device readily useful in a 3D or spatial reference
the present invention provides a mapping of the abovemen
tioned resultant angles, preferably about each of three
orthogonal coordinate axes of the spatial pointer reference
frame, including yaw, pitch and roll angles in a spatial pointer
invention, a 3D pointing device utiliZing a six-axis motion
55
sensor module with an enhanced comparison method for
eliminating accumulated errors of said six-axis motion sensor
module to obtain deviation angles corresponding to move
ments and rotations of said 3D pointing device in a spatial
60
pointer reference frame is provided. The 3D pointing device
and the comparison method provided by the present invention
by comparing signals from the abovementioned six-axis
motion sensor module capable of detecting rotation rates or
frame. Furthermore, there is a need to provide an enhanced
comparison method applicable to the processing of signals of
angular velocities of the 3D pointing device about all of the
motion sensors such that errors and/ or noises associated with
XP, YP and ZP axes as well as axial accelerations of the 3D
pointing device along all of the XP, YP and ZP axes. In other
such signals or fusion of signals from the motions sensors
may be corrected or eliminated. In addition, according to the
?eld of application, such output of deviation in 3D reference
65
words, the present invention is capable of accurately output
ting the abovementioned deviation angles including yaw,
US 8,441,438 B2
5
6
pitch and roll angles in a 3D spatial pointer reference frame of
According to another embodiment of the present invention,
a method for obtaining a resulting deviation including result
the 3D pointing device to eliminate or reduce accumulated
errors and noises generated over time in a dynamic environ
ment including conditions such as being subject to a combi
ant angles in a spatial pointer reference frame of a three
dimensional (3D) pointing device utiliZing a six-axis motion
nation of continuous movements, rotations, external gravity
sensor module therein and subject to movements and rota
forces and additional extra accelerations in multiple direc
tions in dynamic environments in said spatial pointer refer
tions or movement and rotations that are continuously non
ence frame is provided. Said method comprises the steps of:
obtaining a previous state associated With previous angular
velocities 00x, my, 002 gained from the motion sensor signals of
linear With respect to time; and furthermore, based on the
deviation angles being compensated and accurately outputted
in 3D spatial pointer reference frame may be further mapped
the six-axis motion sensor module at a previous time T-l;
obtaining a current state of the six-axis motion sensor module
onto or translated into another reference frame such as the
abovementioned display frame, for example a reference in
by obtaining measured angular velocities 00x, my, 002 gained
tWo-dimension (2D).
from the motion sensor signals at a current time T; obtaining
a measured state of the six-axis motion sensor module by
According to another example embodiment of the present
invention, a 3D pointing device utiliZing a six-axis motion
obtaining measured axial accelerations Ax, Ay, AZ gained
sensor module is provided; Wherein the six-axis motion sen
sor module of the 3D pointing device comprises at least one
gyroscope and at least one accelerometer. In one preferred
from the motion sensor signals at the current time T and
calculating predicted axial accelerations Ax', Ay', AZ' based
embodiment of the present invention, the six-axis motion
sensor module comprises a rotation sensor capable of detect
20
ing and generating angular velocities of 00x, my, 002 and an
accelerometer capable of detecting and generating axial
accelerations of Ax, Ay, AZ. It can be understood that in
another preferred embodiment, the abovementioned rotation
sensor may comprise three gyroscopes corresponding to each
of the said angular velocities of 00x, my, 002 in a 3D spatial
25
According to another aspect of the present invention, a
method for mapping deviation angles associated With move
30
the 3D pointing device. The rotation sensor detects the rota
tion of the 3D pointing device With respect to a reference
frame associated With the 3D pointing device and provides a
angles including yaW, pitch and roll angles in a spatial pointer
35
reference frame to an pointing object, such as a pointer, hav
ing movements in a display frame, preferably a 2D reference
frame, comprises the steps of obtaining boundary informa
tion of the display frame by calculating a prede?ned sensitiv
ity associated With the display frame and performing angle
the reference frame, namely, Xp, Yp and Zp of the 3D spatial
pointer frame. The accelerometer detects the axial accelera
tions of the 3D pointing device With respect to the spatial
ments and rotations of a 3D pointing device in a spatial
pointer reference frame onto a display frame of a display
having a predetermined screen siZe is provided. In one
embodiment, the method for mapping or translating deviation
rotation rate or angular velocity output. The angular velocity
output includes three components corresponding to the rota
tion rate or angular velocities 00x, my, 002 of the 3D pointing
device about the ?rst axis, the second axis and the third axis of
module to said resulting deviation comprising said resultant
angles in said spatial pointer reference frame of the 3D point
ing device.
pointer reference frame of the 3D pointing device; Whereas
the abovementioned accelerometer may comprise three
accelerometers corresponding to each of the said axial accel
erations Ax, Ay, AZ in a 3D spatial pointer reference frame of
on the measured angular velocities 00x, my, 002 of the current
state; obtaining an updated state of the six-axis motion sensor
module by comparing the current state With the measured
state of the six-axis motion sensor module; and calculating
and converting the updated state of the six axis motion sensor
40
and distance translation in the display frame based on said
deviation angles and boundary information.
pointer reference frame such as a 3D-pointer reference frame
and provides an acceleration output. The acceleration output
BRIEF DESCRIPTION OF THE DRAWINGS
includes three components corresponding to the accelera
tions, Ax, AZ, Ay of the 3D pointing device along the ?rst axis,
the second axis and the third axis of the reference frame,
45
The accompanying draWings are included to provide a
further understanding of the invention, and are incorporated
in and constitute a part of this speci?cation. The draWings
illustrate embodiments of the invention and, together With the
description, serve to explain the principles of the invention.
50
FIG. 1 shoWs a knoWn related art having a 5-axis motion
sensor in 2D reference frame.
namely, Xp,Yp and Zp of the 3D spatial pointer frame. It can,
hoWever, be understood that the axes of Xp, Yp and Zp of the
3D spatial pointer frame may too be represented simply by the
denotation of X, Y and Z.
According to another example embodiment of the present
invention, a method for compensating accumulated errors of
signals of the abovementioned six-axis motion sensor module
FIG. 2 shoWs the knoWn related art having a 5-axis motion
sensor as shoWn in FIG. 1 being rotated or rolled about Xp
in dynamic environments associated in a spatial pointer ref
erence frame is provided. In one embodiment, the method
may be performed or handled by a hardWare processor. The
processor is capable of compensating the accumulated errors
associated With the resultant deviation in relation to the sig
nals of the above-mentioned six-axis motion sensor module
of the 3D pointing device subject to movements and rotations
in a spatial pointer reference frame and in a dynamic envi
axis and is subject to further dynamic interactions or environ
55
FIG. 3 is an exploded diagram shoWing a 3D pointing
device utiliZing a six-axis motion sensor module according to
one embodiment of the present invention in a 3D spatial
60
ronment by performing a data comparison to compare signals
of rotation sensor related to angular velocities With the ones of
accelerometer related to axial accelerations such that the
resultant deviation corresponding to the movements and rota
tions of the 3D pointing device in the 3D spatial pointer frame
may be obtained accurately over time in the dynamic envi
ronments.
ment.
65
pointer reference frame.
FIG. 4 is a schematic block diagram illustrating hardWare
components of a 3D pointing device according to one
embodiment of the present invention.
FIG. 5 is a schematic diagram shoWing a 3D pointing
device utiliZing a six-axis motion sensor module according to
anther embodiment of the present invention in a 3D spatial
pointer reference frame.
US 8,441,438 B2
8
7
FIG. 6 is an exploded diagram showing a 3D pointing
orthogonal coordinate axes XPYPZP of the spatial pointer
device utilizing a six-axis motion sensor module according to
anther embodiment of the present invention in a 3D spatial
reference frame. The angular velocities 00x, my and 002 are
pointer reference frame.
ing deviation angles of a 3D pointing device having move
ments and rotations in a 3D spatial pointer reference frame
tively. The accelerometer 344 detects and generates the sec
ond signal set including axial accelerations Ax, Ay, AZ asso
ciated With the movements and rotations of the 3D pointing
device 300 along each of the three orthogonal coordinate axes
and in a dynamic environment according to an embodiment of
XPYPZP of the spatial pointer reference frame. The axial
the present invention.
accelerations Ax, Ay and AZ are corresponding to the coordi
nate axes XP, YP and ZP respectively. The term “six-axis”
means the three angular velocities 00x, my, 002 and the three
axial accelerations Ax, Ay, AZ. It can therefore be understood
corresponding to the coordinate axes XP, YP and ZP respec
FIG. 7 is a How chart illustrating a method for compensat
FIG. 8 shoWs a How chart illustrating a method of mapping
deviation angles of a 3D pointing device having movements
and rotations in a 3D spatial pointer reference frame and in a
dynamic environment onto a display reference frame accord
ing to another embodiment of the present invention.
FIG. 9 is a schematic diagram shoWing the mapping of the
resultant angles of the resultant deviation of a 3D pointing
device according to an embodiment of the present invention.
that the abovementioned six axes of XPYPZP may not need to
be orthogonal in a speci?c orientation and they may be rotated
in different orientations; the present invention discloses such
coordinate system for illustrative purposes only and any coor
dinates in different orientation and/or denotations may too be
possible.
The data transmitting unit 346 is electrically connected to
DESCRIPTION OF THE EMBODIMENTS
20
Reference Will noW be made in detail to the present
embodiments of the invention, examples of Which are illus
trated in the accompanying draWings. Wherever possible, the
same reference numbers are used in the draWings and the
description to refer to the same or like parts.
25
FIG. 3 is an exploded diagram shoWing a 3D pointing
device 300 according to an embodiment of the present inven
tion. The 3D pointing device 300 is subject to movements and
rotations in dynamic environments in a 3D spatial pointer
reference frame. The spatial pointer reference frame is analo
30
gous to the reference frame XPYPZP in FIG. 1 and FIG. 2. The
movements and rotations of the 3D pointing device 300 in the
aforementioned dynamic environments in the spatial pointer
reference frame may be continuously nonlinear With respect
to time.
The 3D pointing device 300 includes a top cover 310, a
35
spatial pointer reference frame, of the resulting deviation of
the six-axis motion sensor module 302 of the 3D pointing
device 300 is obtained under the aforementioned dynamic
environments and such that it is preferably obtained and out
putted in an ab solute manner re?ecting or associating With the
actual movements and rotations of the 3D pointer device of
45
the present invention in said spatial pointer reference frame.
In addition, said comparison utiliZed by the computing pro
ting unit 346, and the computing processor 348 may be all
having a longitudinal side con?gured to be substantially par
allel to the longitudinal surface of the housing 330. An addi
tional battery pack 322 provides electrical poWer for the
entire 3D pointing device 300.
50
accelerations Ax, Ay, AZ. The abovementioned measured
state may include a measurement of said second signal set or
measured Ax, Ay, AZ and a predicted measurement ofAx', Ay'
55
device 300 includes a six-axis motion sensor module 302 and
a processing and transmitting module 304. The six-axis
60
provided in the later content.
In this embodiment, the computing processor 348 of the
processing and transmitting module 304 further includes a
resulting deviation in the spatial pointer reference frame to a
movement pattern in a display reference frame different from
the spatial pointer reference frame. The display reference
The rotation sensor 342 of the six-motion sensor module
velocities 00x, my, 002 associated With the movements and rota
tions of the 3D pointing device 300 about each of three
and AZ' obtained based on or calculated from the ?rst signal
set. Details of different states of the six-axis motion sensor
module of the 3D pointing device of the present invention are
mapping program for translating the resultant angles of the
and the accelerometer 344. The processing and transmitting
module 304 includes the data transmitting unit 346 and the
computing processor 348.
302 detects and generates the ?rst signal set including angular
cessor 348 may further comprise an update program to obtain
an updated state of the six-axis motion sensor module based
on a previous state associated With a ?rst signal set in relation
to the angular velocities cox, coy, oZ and a measured state
associated With said second signal set in relation to the axial
FIG. 4 is a schematic block diagram illustrating hardWare
components of the 3D pointing device 300. The 3D pointing
motion sensor module 302 includes the rotation sensor 342
FIG. 1 and FIG. 2. In order to calculate the resulting devia
tion, the computing processor 348 utiliZes a comparison to
eliminate accumulated errors of the ?rst and second signal
sets of the six-axis motion sensor module 302, Whereby the
40
one embodiment, the housing 330 may comprise the top
attached to the PCB 340. The PCB 340 is enclosed by the
housing 330. The PCB 340 includes at least one substrate
of the 3D pointing device 300 including three resultant angles
preferably about each of the three axes of the spatial pointer
reference frame. The resultant angles include the yaW angle
111, the pitch angle 112 and the roll angle 113 as shoWn in
erably about each of three orthogonal coordinate axes of the
cover 310 and the bottom cover 320. The housing 330 may
move and rotate in the spatial pointer reference frame accord
ing to user manipulation or any external forces in any direc
tion and/ or under the abovementioned dynamic environ
ments. As shoWn in the FIG. 3, in one embodiment, the
rotation sensor 342, the accelerometer 344, the data transmit
?rst and second signal sets. The data transmitting unit 346
transmits the ?rst and second signal sets of the six-axis
motion sensor module 302 to the computing processor 348
via electronic connections on the PCB 340. The computing
processor 348 receives and calculates the ?rst and second
signal sets from the data transmitting unit 346. The comput
ing processor 348 further communicates With the six-axis
motion sensor module 302 to calculate the resulting deviation
resultant angles in the spatial pointer reference frame, pref
printed circuit board (PCB) 340, a rotation sensor 342, an
accelerometer 344, a data transmitting unit 346, a computing
processor 348, a bottom cover 320, and a battery pack 322.
The top cover 310 may include a feW control buttons 312 for
a user to issue prede?ned commands for remote control. In
the six-axis motion sensor module 302 for transmitting the
65
frame is analogous to the reference frame XDYDZD in FIG. 1
and FIG. 2. The movement pattern may be displayed on a
screen of a 2D display device similar to the display device 120
US 8,441,438 B2
9
10
in FIG. 1 and FIG. 2. The mapping program translates the
ment, the computing processor 554 also performs mapping by
translating the resultant angles of the resulting deviation of
the 3D pointing device in the spatial pointer reference frame,
resultant angles, preferably about each of three orthogonal
coordinate axes of the spatial pointer reference frame to the
movement pattern according to a sensitivity input correlated
to the display reference frame.
FIG. 5 is a schematic diagram shoWing a 3D pointing
preferably about each of three orthogonal coordinate axes of
the spatial pointer reference frame, to a movement pattern in
a display reference frame associated With the notebook com
puter 580. The movement pattern may be displayed on the
screen 582 of the notebook computer 580.
FIG. 6 is an exploded diagram shoWing a 3D pointing
device 500 utiliZing a six-axis motion sensor module accord
ing to anther embodiment of the present invention in a 3D
spatial pointer reference frame. As shoWn in FIG. 5, the 3D
pointing device 500 may comprise tWo parts 560 and 570 in
device 600 utiliZing a six-axis motion sensor module accord
ing to anther embodiment of the present invention in a 3D
data communication With each other. In one embodiment, the
?rst part 560 includes a top cover (not shoWn), a PCB 540, a
six-axis motion sensor module 502 comprising a rotation
sensor 542 and an accelerometer 544, a data transmitting unit
546, a bottom cover 520, and a battery pack 522. The data
spatial pointer reference frame. The 3D pointing device 600
may further comprises a built-in display 682. In other Words,
the abovementioned display reference frame associated With
a display may need not to be external to the spatial pointer
reference frame in terms of the hardWare con?guration of the
present invention. In one embodiment, the 3D pointing device
600 comprises a bottom cover 620, a PCB 640, a battery pack
transmitting unit 546 transmits the ?rst signal set (00x, my, 002)
generated by the rotation sensor 542 of the six-motion sensor
module 502 and the second signal set (Ax, Ay, AZ) generated
by the accelerometer 544 of the six-motion sensor module
502 to the data receiving unit 552 of the second part 570 via
Wireless communication or connection including Wireless
local area netWork (WLAN) based on IEEE 802.1 1 standards
or BluetoothTM. It can be understood that in another embodi
ment, Wired communication or connection via a physical
cable or electrical Wires connecting the ?rst part 560 and the
622, a rotation sensor 642, an accelerometer 644, a data
20
682, and a top cover 610. LikeWise, in one embodiment, the
housing 630 may comprise the top and bottom covers 610,
25
620. A built-in display 682 may too be integrated on the
housing 630; the six-axis motion sensor module 602 may
comprise the rotation sensor 642 and the accelerometer 644.
30
The data transmitting unit 646 and the computing processor
648 may also be integrated as a processing and transmitting
module 604 of the 3D pointing device 600.
The computing processor 648 of the processing and trans
mitting module 604 may too perform the mapping of resultant
second part 570 may too be possible.
In one embodiment, the second part 570 may be an external
processing device to be adapted to another electronic com
puting apparatus or system such as a personal computer 580;
for instance, the second part 570 may be coupled or adapted
deviation from or in said spatial reference frame or 3D refer
ence frame to a display reference frame such as a 2D refer
to an laptop computer via a standard interface, such as the
universal serial bus (USB) interface depicted as shoWn in
FIG. 5. The ?rst part 560 and the second part 570 communi
cate via the data transmitting unit 546 and the data receiving
unit 552. As previously mentioned, the data transmitting unit
35
546 and the data receiving unit 552 may communicate
through Wireless connection or Wired connection. In other
Words, in terms of hardWare con?guration and data transmis
sion, in one embodiment of the present invention, the six-axis
motion sensor module 502 comprising the rotation sensor 542
transmitting unit 646, a computing processor 648, a display
ence frame by translating the resultant angles of the resulting
deviation of the 3D pointing device 600 in the spatial pointer
reference frame, preferably about each of three orthogonal
coordinate axes of the spatial pointer reference frame to a
movement pattern in a display reference frame associated
With the 3D pointing device 600 itself. The display 682 dis
plays the aforementioned movement pattern. The top cover
40
610 includes a transparent area 614 for the user to see the
and the accelerometer 544 may be disposed distally from the
processing unit or computing processor 554; the signals from
display 682.
the six-axis motion sensor module 502 may then be transmit
obtaining and/or outputting a resulting deviation including
FIG. 7 is an explanatory ?oW chart illustrating a method for
ted via the data transmitting units 546, 552 to the computing
processor 554 via Wired or Wireless communication including
for example IEEE 802.11 standards or BluetoothTM.
The second part 570 of the 3D pointing device 500 accord
ing to one embodiment of the present invention comprises the
data transmitting unit 552 and the processor 554. The data
transmitting unit 552 of the second part 570 may be in data
communication With the other data transmitting unit 546 dis
posed distally therefrom in the ?rst part 560 as previously
mentioned. The data transmitting unit 552 in the second part
570 receives the ?rst and second signal sets from the data
transmitting unit 546 in the ?rst part 560 and transmits the
?rst and second signal sets to the computing processor 554. In
one embodiment, the computing processor 554 performs the
aforementioned calculation as Well as comparison of signals.
In one embodiment, said comparison utiliZed by the comput
ing processor 554 may further comprise an update program to
45
spatial pointer reference frame and in dynamic environments
according to an embodiment of the present invention. The
method in FIG. 7 may be a program or comparison model to
be embedded or performed by the processing unit or comput
50
55
60
subject to movements and rotations in dynamic environments
in said spatial pointer reference frame is provided; and said
method may comprise the folloWing steps. First of all, as
shoWn in FIG. 7, different states including “previous state”,
“current state”, “measured state” and “update state” of the
six-axis motion sensor module may be provided to represent
a step or a set of steps utiliZed by the method for obtaining the
said second signal set. The measured state may further
set. The computing processor 554 is external to the housing of
the 3D pointing device as depicted in FIG. 5. In one embodi
ing processor 348, 554, 648 of the processing and transmit
ting module according to different embodiments of the
present invention recited herein for illustrative purposes.
Accordingly, in one embodiment of the present invention,
a method for obtaining a resulting deviation including result
ant angles in a spatial pointer reference frame of a 3D pointing
device utiliZing a six-axis motion sensor module therein and
obtain an updated state based on a previous state associated
With said ?rst signal set and a measured state associated With
include a measurement of said second signal set and a pre
dicted measurement obtained based on the ?rst second signal
resultant angles in a spatial pointer reference frame of a 3D
pointing device having movements and rotations in a 3D
resulting deviation in 3D reference frame, and preferably in
65
the above-mentioned absolute manner. In one exemplary
embodiment, the method comprises obtaining a previous
state of the six-axis motion sensor module (such as steps 705,
US 8,441 ,438 B2
11
12
710); and wherein the previous state includes an initial -value
set or a ?rst quatemion associated With previous angular
velocities 00x, my, 002 gained from the motion sensor signals of
the six-axis motion sensor module at a previous time T-l;
continuous loop betWeen a previous time frame T-l and a
present time frame T; details on the replacement of the ?rst
quatemion at T-l With the later outputted quatemion at T is to
be provided in the later content. It can be understood that one
may make reference to Euler Angles for dentition on quater
obtaining a current state of the six-axis motion sensor module
by obtaining measured angular velocities 00x, my, 002 gained
nion. Similarly, it can be easily comprehended that the above
mentioned previous time T-l and present time T may too be
from the motion sensor signals of the six-axis motion sensor
module at a current time T (such as steps 715, 720); obtaining
a measured state of the six-axis motion sensor module by
obtaining measured axial accelerations Ax, Ay, AZ gained
from the motion sensor signals of the six-axis motion sensor
module at the current time T (such as step 725) and calculat
ing predicted axial accelerations Ax', Ay', AZ' based on the
measured angular velocities 00x, my, 002 of the current state of
the six-axis motion sensor module (such as step 73 0); obtain
ing an updated state of the six-axis motion sensor module by
comparing the current state With the measured state of the
six-axis motion sensor module (such as step 735); and calcu
5
be performed simultaneously, such as the obtaining of signals
lating and converting the updated state of the six axis motion
sensor module to said resulting deviation comprising said
20
resultant angles in said spatial pointer reference frame of the
3D pointing device (745). In order to provide a continuous
loop, the result of the updated state of the six-axis motion
sensor module may preferably be outputted to the previous
state; in one embodiment, the updated state may be a quater
nion, namely third quaternion as shoWn in the ?gure, such that
it may be directly outputted to the abovementioned previous
state of another quaternion, namely the abovementioned ?rst
quaternion and as shoWn in the ?gure (such as step 740).
In addition, it can be understood that the abovementioned
25
30
from the six-axis motion sensor module may be performed
simultaneously instead of one after another. It can therefore
be understood that the steps recited herein are for illustrative
purposes only and any other sequential orders or simulta
neous steps are possible and shall be Within the scope of the
present invention. When step 710 is performed for the ?rst
time, the ?rst quaternion initialiZed in step 705 is obtained.
Otherwise, the ?rst quatemion used in the present time T is
generated in the previous time T-l. In other Words, the step
710 may generally refer to or represented by the abovemen
tioned “previous state” of the six-axis motion sensor module;
according to another embodiment, the previous state may
refer to the steps of 705 and 710.
comparison utiliZed by the processing and transmitting mod
The next may be to obtain the ?rst signal set generated by
the rotation sensor, Which includes the measured angular
velocities 00x, my and 002 as shoWn in step 715 according to an
ule and comprising the update program may too make refer
ence to said different states of the six-axis motion sensor
module as shoWn in FIGS. 7 and 8. As mentioned previously,
the update program may be utiliZed by the processor to obtain
the updated state of the six-axis motion sensor module based
on the previous state associated With a ?rst signal set in
relation to the angular velocities 00x, my, 002 and the measured
state associated With said second signal set in relation to the
axial accelerations Ax, Ay, AZ. The abovementioned mea
sured state may include a measurement of said second signal
set or measured Ax, Ay, AZ and a predicted measurement of
Ax', Ay' and AZ' obtained based on or calculated from the ?rst
signal set. Details of each of the abovementioned states of the
six-axis motion sensor module and the related steps of the
substitute by a present time T and a next time T+l respectively
and shall too fall Within the scope and spirit of the present
invention.
The ?rst quatemion With respect to the previous time T is
obtained as shoWn in the ?gure as step 710. The method
illustrated in FIG. 7 may be performed in consecutive time
frames. According to one embodiment of the present inven
tion, steps 710-745 may be in a loop that may be performed
one step at a time. In another embodiment, multiple steps may
35
exemplary embodiment of the present invention. In step 720,
the second quaternion With respect to a present time T is
calculated and obtained based on the angular velocities 00x, my
and 002. The step 715 and 720 may generally refer to or may be
represented by the abovementioned “current state” of the
40
six-axis motion sensor module. In one embodiment, the com
puting processor may use a data conversion utility to convert
the angular velocities 00x, my and 002 into the second quater
nion. This data conversion utility may be a program or
instruction represented by the folloWing equation (1).
45
method for obtaining the resulting deviation of the 3D point
ing device in 3D reference frame are as folloWs.
Referring to FIG. 7 again, the method for obtaining a
resulting deviation including resultant angles in a spatial
pointer reference frame of 3D pointing device utiliZing a
50
six-axis motion sensor module according to one embodiment
of the present invention may begin at the obtaining of a
previous state of the six-axis motion sensor module. In one
embedment, the previous state of the six-axis motion sensor
module may preferably be in a form of a ?rst quaternion, and
55
the ?rst quaternion may be preferably initialiZed (step 705) at
a very beginning of the process or method and as part of the
obtaining of the previous state thereof. In other Words,
according to one embodiment of the present invention, the
signals of the six-axis motion sensor are preferably to be
initialiZed to Zero and in particular, the signal or value asso
60
motion sensor module according to one embodiment of the
predetermined initial values. Alternatively, the ?rst quater
generated by the rotation sensor and the accelerometer at a
next time frame such that the method as shoWn in FIG. 7 is a
the differential equation (1).
As shoWn in the ?gure, the “measured state” of the six-axis
ciated With the yaW angle in terms of a quatemion value. The
four elements of the ?rst quatemion may be initialiZed With
nion may be initialiZed or replaced by another signal sets
Equation (1) is a differential equation. The quatemion on
the left side of the equal sign is the ?rst order derivative With
respect to time of the quaternion (qO, ql, q2, q3) on the right
side of the equal sign. The data conversion utility uses the ?rst
quatemion as the initial values for the differential equation (1)
and calculates the solution of the differential equation (1).
The second quatemion may be represented by a solution of
65
present invention may generally refer or may be represented
by steps 725 and 730. In step 725, the second signal set
generated by the accelerometer may be obtained, Which
includes measured axial accelerations Ax, Ay and AZ; or Ax,
Ay and AZ may refer to the measurement of the axial accel
US 8,441,438 B2
13
14
erations obtained. In order to obtain said measured state of the
accelerations of accelerometers and current state may be
obtained based on an exemplary equation of:
six-axis motion sensor of the present invention, according to
one embodiment, predicted axial accelerations Ax', Ay' and
AZ' may too be calculated and obtained based on the above
mentioned current state of the six-axis motion sensor module
Preferably, a second probability (measurement probability)
or the second quaternion as shoWn in step 73 0. In other Words,
associated With the measured state may be further obtained
based on an exemplary equation of:
tWo sets of axial accelerations may be obtained for the mea
sured state of the six-axis motion sensor module; one may be
the measured axial accelerations Ax, Ay, As in step 725 and
the other may be the predicted axial accelerations Ax', Ay', AZ'
in step 730 calculated based on the abovementioned current
state or second quaternion in relation to the measured angular
velocities thereof. Furthermore, in one embodiment, the com
puting processor may use a data conversion utility to convert
(10)
the measured axial accelerations Ax, Ay and AZ into a quater
nion. This data conversion utility may be a softWare program
Wherein RtImeasurement noise
represented by the folloWing equations (2), (3) and (4).
As an illustrative example, the abovementioned ?rst and sec
ond probabilities may be further utiliZed to obtain the updated
state of the six-axis motion sensor module based on an exem
20
plary method of data association of an exemplary equation of:
25
In one embodiment, the result of the updated state of the
six-axis motion sensor module, preferably involving com
(4)
The computing processor calculates the solution (qO, ql, q2,
q3) of the equations (2), (3) and (4).
According to an exemplary embodiment of the method for
obtaining a resulting deviation including resultant angles in a
spatial pointer reference frame of a 3D pointing device uti
liZing a six-axis motion sensor module, it may be preferable
resulting deviation including resultant angles in a spatial
to compare the current state of the six-axis motion sensor
module With the measured state thereof With respect to the
30
present time frame T by utiliZing a comparison model. In
other Words, in one embodiment as shoWn in step 735, it is
preferable to compare the second quaternion in relation to the
measured angular velocities of the current state at present
time T With the measured axial accelerations Ax, Ay, AZ as
Well as the predicted axial accelerations Ax', Ay', AZ' also at
present time T. FolloWing Which, a result may be obtained as
35
an updated state of the six-axis motion sensor module. In an
40
module at preset time T. Instructions including equations
related to the abovementioned current state, measured state
and updated state may be illustrated in the folloWing.
According to an exemplary embodiment of the comparison
model utiliZed by the present invention in relation to step 735
pointer reference frame in the folloWing steps as shoWn in the
?gure. It can be understood that the examples of current state,
measured state, state update, data association and probabili
explanatory example, the updated state may generally refer to
the update of the current state of the six-axis motion sensor
parison or data association represented by the equations, may
be a third quaternion as shoWn in the ?gure. Furthermore, the
result may then be further outputted and utiliZed to obtain a
45
ties of the comparison model and method of the present
invention are provided for illustrative purposes only.
As mentioned previously, it may be preferable to output the
result of the updated state, preferably in a form of third
quaternion, to the previous state of the six-axis motion sensor
module as shoWn in step 740 in the ?gure. In other Words, in
one embodiment, the ?rst quaternion may be replaced by the
abovementioned third quaternion or substitute directly any
previous values of ?rst quaternion in the previous time T for
further process in a loop. In other Words, the third quaternion
With respect to the present time T becomes the ?rst quaternion
With respect to the next time such as T+l ; or, the third quater
nion at previous time frame T-l outputted may noW be the
?rst quaternion at present time frame T.
In step 745, the updated state of the six-axis motion sensor
module of the present invention may be further calculated and
convert to the resulting deviation including resultant angles
as shoWn in the ?gure, the current state correlated to the
abovementioned second quaternion and in relation to the
angular velocities of gyroscope(s) may be obtained based on
an exemplary equation of:
50
associated With the spatial pointer reference frame, Wherein
the resultant angles includes the yaW angle, pitch angle and
roll angle of the 3D pointing device associated With the spatial
pointer reference frame, preferably about each of three
orthogonal coordinate axes of the spatial pointer reference
55
frame. In one embodiment, the computing processor may use
a data conversion utility to convert the third quaternion of the
updated state of the six-axis motion sensor module into the
Preferably, a ?rst probability (state transition probability)
associated With the said current state may be further obtained
based on an exemplary equation of:
yaW, pitch and roll angles thereof. This data conversion utility
FX : a?xH, 14!)
635141
F” : ?fwii, 14!)
may be a program or instruction represented by the folloWing
(6)
60
equations (l2), (l3) and (14).
(7)
1314,
Z(qoq3 +4142)
(12)
yaW : arctan W
‘10 + £11 — £12 — £13
Wherein Qfadditional motion noise
Likewise, the measured state correlated to the abovemen
tioned second axial accelerations and in relation to the axial
65
Pitch = aICSiIIQUIoQz — £13 £11 ))
(13)
US 8,441,438 B2
15
16
-continued
display to be mapped With deviation to a movement pattern in
14
roll : arctan[
2D display reference frame and the 3D pointing device of the
( )
present invention outputted With said deviation including
yaW, pitch and roll angles in 3D pointer reference frame;
Wherein the relationship may be a distance relationship. In
The variables qO, ql, q2 and q3 in equations (12), (13) and (14)
another embodiment, the sensitivity input may be a display
screen siZe including boundary information predetermined
are the four elements of the third quatemion.
For a looped method continuous With respect to time, in
one embodiment of the present invention, the method utilized
by a user; Wherein the boundary information may be obtained
based on a user input or manual input data from the user. In
by for example the computing processor communicated With
still another embodiment, the sensitivity input may be pre
the six-axis motion sensor module may return to step 710 to
de?ned or preset in the mapping program such that the param
eter of the sensitivity input is a preset value for either increase
perform the comparison process or method With respect to the
next time T+1. In addition, the abovementioned resulting
or decrease the movement patterns including distance or
deviation including resultant angles comprising yaW, pitch
and roll angles in the spatial reference frame converted from
the third quatemion is preferably obtained and outputted in an
absolute manner re?ecting or associating With the actual
movements and rotations of the 3D pointer device of the
present invention in said spatial pointer reference frame. It
can be understood that said actual movements and rotations of
20
the 3D pointer device of the present invention in the spatial
pointer reference frame or 3D reference frame may refer to
real-time movements and rotations associated With vectors
having both magnitudes and directions along or about
orthogonal axes in the spatial pointer reference frame under
the dynamic environments.
the 3D pointing device 930 is pointing at. The boundary point
25
FIG. 8 shoWs a How chart illustrating a method of mapping
resultant deviation angles of a 3D pointing device having
movements and rotations in a 3D spatial pointer reference
frame and in a dynamic environment onto a display reference
frame according to another embodiment of the present inven
tion. FIG. 9 is a schematic diagram showing the aforemen
tioned mapping of the resultant angles of the resultant devia
tion of a 3D pointing device according to this embodiment.
For illustrative purposes, the difference betWeen FIG. 7 and
FIG. 8 may be represented by the additional mapping step 750
number of pixels to be moved or mapped from said deviation
of the 3D pointing device.
FIG. 9 is a bird’s-eye vieW ofa 3D pointing device 930 and
the display screen 910 of a display device according to an
embodiment of the present invention. The display screen has
a central point 922, a target point 924 and a boundary point
926. The central point 922 is the geometric center of the
display screen 910. The target point 924 is the position that
30
926 is a point on the right boundary of the display screen 910.
The points 922, 924, 926 and the 3D pointing device 930 are
on a common plane parallel to both the XD axis and the ZD
axis of the display reference frame XDYDZD. Virtual beams
942, 944 and 946 are imaginary light beams from the 3D
pointing device 930 to the central point 922, the target point
924 and the boundary point 926, respectively. The distance P
is the distance betWeen the central point 922 and the target
point 924, While the distance Pmax is the distance betWeen the
central point 922 and the boundary point 926. The distance d
processor may include a mapping program that performs the
is the distance betWeen the central point 922 and the 3D
pointing device 930. The aforementioned yaW angle of the
resultant deviation of the 3D pointing device 930 is the angle
6 betWeen the virtual beams 942 and 944, While the angle
6m,C is the angle betWeen the virtual beams 942 and 946. The
aforementioned mapping area is a plane including the display
surface of the display screen 910 in the display reference
mapping step 750. At step 750, the processing and transmit
ting module may obtain display data including for example,
frame. The display surface of the display screen 910 is a
subset of the mapping area.
35
as shoWn in FIG. 8. Steps 705-745 in FIG. 8 are the same as
their counterparts in FIG. 7, Which perform the comparison
process for the 3D pointing device. Step 750 performs the
mapping process for the 3D pointing device. The computing
40
display screen siZe such as boundary information, and trans
lates the resultant angles of the resulting deviation associated
45
With the spatial pointer reference frame, preferably about
sensitivity [3 is de?ned by the folloWing equation (15).
each of three orthogonal coordinate axes of the spatial pointer
reference frame, to a movement pattern in a mapping area in
a display reference frame based on a sensitivity input corre
lated to the display reference frame. It can be understood that
the above-mentioned display data may too include or refer to
In this embodiment, the aforementioned sensitivity input is
provided by the user of the 3D pointing device 930. The
50
Pmax
9%
the type of display such as LED, LCD, touch panel or 3D
(15)
display as Well as frequency rate of display such as 120 HZ or
240 HZ. In one embodiment, the display reference frame
associated With the display to be mapped may be a 2D display
55
reference frame; in another embodiment, the display refer
The variable [3 in equation (16) is the sensitivity input
de?ned by user.
ence frame may be a 3D display reference frame of a 3D
display.
The aforementioned display data may further include a
sensitivity input. The aforementioned sensitivity input is a
60
parameter Which may be inputted and adjusted by a user
through control buttons attached on the housing of the 3D
pointing device. The sensitivity input may represent the sen
sitivity of the display device With respect to the movement of
the 3D pointing device. For details of the mapping process,
please refer to FIG. 9. In one embodiment, the sensitivity
input is a parameter representing the relationship betWeen the
The folloWing equation (16) may be derived from equation
(15) and geometry.
(16)
65
US 8,441,438 B2
17
18
The following equation (17) may be derived from equa
In vieW of the above, it is clear that such obtaining and
outputting of deviation including 3D angles in a spatial
tions (16).
pointer reference frame in an “absolute” manner of the
Pm >
Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.
Why Is My Information Online?