Speech Recognition Copilot for the A2A B-17

A Living Legend
lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Speech Recognition Copilot for the A2A B-17

Post by lequinne »

Hi y'all -

The A2A B-17 is still my favorite plane to fly. I got tired of reaching for the flap and gear switches, primer etc., so I made a copilot add-on for the B17, based on Microsoft Speech Recognition. It controls a bunch of things on voice command including:

Fuel Pumps
Filters
Engine Start procedures
Emergency procedures
Props (just call out an RPM)
Gear
Flaps (by fractions or percent)
Oil Dilution
Tail wheel lock
Hand pump
C1 AP
APU
Tokyo valves
Fuel Transfer
and perhaps some other things I'm forgetting?

In any case it's all in the manual, and I'll add more if people seem to be interested.

Download the latest version (1.2.0) of the DLL and PDF manual from here:
https://www.dropbox.com/sh/2zf6geqprq11 ... stV1a?dl=0

Kilstorm McBride compiled an Excellent quick reference for use in flight which can be downloaded as a PDF here:
https://www.dropbox.com/s/tghwz661pxeng ... s.pdf?dl=0
and as a pretty sweet vintage version in PSD format here:
https://www.dropbox.com/sh/rrh1bt4me8pd ... zoW-a?dl=0

Cheers,
Farley
Last edited by lequinne on 12 Nov 2015, 22:09, edited 8 times in total.

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

This is very much IN BETA (I've been working on it and using it for a couple months, but no one else has tested it yet) so...

PLEASE:
-->Try this out!!! it's actually taken me no small amount of work, and I hope it can enhance your experience as it has enhanced mine

-->Post if it works! I'd like to hear it!

-->Read the manual, and then post if it doesn't work! We'll figure it out, don't worry.

-->Be as specific as possible! I'll do my utmost to provide A+ support!

-->Post any suggestions for new commands, expanded syntax, tweaks, features you find annoying and want me to get rid of or make optional, anything! I can't promise I'll implement them all, but I'll investigate them all

-->Keep the LOG box checked at all times, it may be helpful to me in the event something goes wrong

-->Bear in mind there is a learning curve when it comes to speech recognition. If it's rough at first, give it some time.

-->If you've given it some time, please let me know if there are any commands which are consistently unrecognized or falsely recognized, I would desperately like to hear about it. Please give me the Exact wording you used, and show me your log file.

I really hope you all enjoy this as much as I enjoyed making it!

Farley

User avatar
Lewis - A2A
A2A Lieutenant Colonel
Posts: 33284
Joined: 06 Nov 2004, 23:22
Location: Norfolk UK
Contact:

Re: Speech Recognition Copilot for the A2A B-17

Post by Lewis - A2A »

Interesting stuff, thanks for posting your work!

cheers,
Lewis
A2A Facebook for news live to your social media newsfeed
A2A Youtube because a video can say a thousand screenshots,..
A2A Simulations Twitter for news live to your social media newsfeed
A2A Simulations Community Discord for voice/text chat

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

No problem Lewis, I really would be honored if you and/or others at A2A can take a second to try it out. I admire you guys immensely and I would really, really appreciate any suggestions, tips, or feedback you have!

Sincerely,
Farley

N602AC
Senior Airman
Posts: 112
Joined: 19 Jun 2012, 17:50

Re: Speech Recognition Copilot for the A2A B-17

Post by N602AC »

I just want to say I can't believe nobody has given any feedback on this. It's AWESOME and I love it. It's almost like FS2Crew for the B17. PLEASE keep up the development on this project. Can't get enough of flying the B17 again now.

A couple suggestions I can think of off the top of my head right now are, to add a "hard mute" function so you can mute the listening for long periods of time as opposed to having to hold down a key.

AND/OR,

the ability to assign a key where it ONLY listens when you're pushing it.

Dan
Image

User avatar
Lewis - A2A
A2A Lieutenant Colonel
Posts: 33284
Joined: 06 Nov 2004, 23:22
Location: Norfolk UK
Contact:

Re: Speech Recognition Copilot for the A2A B-17

Post by Lewis - A2A »

Anyway to make it work with standard English?

thanks,
Lewis
A2A Facebook for news live to your social media newsfeed
A2A Youtube because a video can say a thousand screenshots,..
A2A Simulations Twitter for news live to your social media newsfeed
A2A Simulations Community Discord for voice/text chat

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

Dan -
Thanks for the feedback, I'm really glad you're enjoying it!!!! :) I had fun making it, and I definitely plan to keep up development on it! New version in a few days will absolutely include a "hard mute" button. The reason I didn't put one in before is that I've been developing in a vacuum over here... I've got a hardware switch on my headset itself which I use for this purpose, so I just assumed that Everyone Else in the World had identical equipment. :oops:

As regards your other suggestion, I'll look into it. I need to read up again on the behavior of the real-world interphone system. I don't remember whether the pilot must press the PTT to use the interphone. If he must, then I can just have the PTT button reverse its function when the headphones are on and interphone or call is selected on the jackbox.

Lewis -
My sincere apologies for leaving out the British contingent!! I'm not sure whether it's more practical to make a version usable by both US English and standard English systems, or just to compile two separate versions. In any case, I'll investigate this evening and, barring anything truly unforeseen, I'll post something you can use tonight. That is... tonight in the eastern US. ;)

Farley

User avatar
Lewis - A2A
A2A Lieutenant Colonel
Posts: 33284
Joined: 06 Nov 2004, 23:22
Location: Norfolk UK
Contact:

Re: Speech Recognition Copilot for the A2A B-17

Post by Lewis - A2A »

Look forward to giving it a whirl 8)
A2A Facebook for news live to your social media newsfeed
A2A Youtube because a video can say a thousand screenshots,..
A2A Simulations Twitter for news live to your social media newsfeed
A2A Simulations Community Discord for voice/text chat

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

Version 1.1.0 is up! it's just two tiny changes, plus it fixes a rare CTD bug, and it addresses the sub-language issue:

So.....
It turns out Windows languages are a bit silly, but at this point I think I understand what's going on. It seems that with certain licenses of Windows it's impossible to really change your language setting from its default, you can only superficially change it.

So to make a long story short, if my theory is correct, the recognition engine should now load on any type of English language system, provided that the language set in the Speech Recognition options matches the default system language.

The catch is that, due to the Windows limitation outlined above, I am unable to actually test it in anything other than English(US). So I've uploaded it anyway, in the hopes that my theory is correct.


So:
Please, Lewis or anyone else with a non-US English system, try out the new version and post a quick yes if it loads without showing you the "Unable to load Grammar..." error box.

If it still doesn't work, my next guess is that either your default system language is actually US English or some other language despite all indications to the contrary, or that FSX itself is somehow messing with the language setting. It should be possible to work around either of these by setting your SR language to English(US) when you want to use the copilot.

Well, I await the verdict. :?

Hopefully,
Farley

User avatar
Kilstorm
Technical Sergeant
Posts: 555
Joined: 08 Sep 2009, 22:00

Re: Speech Recognition Copilot for the A2A B-17

Post by Kilstorm »

I really enjoy flying with FS2Crew so I am looking for to installing this tonight and giving it a whirl. Where do I put the dll file and has it been tested in P3Dv3 yet?
i9-13900K O/C | ASUS ROG Maximus Z790 Hero | 32GB DDR5 6000MHz Kingston FURY | RTX 4090 24GB | 2x SSD M.2 (2TB Samsung 990 PRO) 1x SSD (4TB Samsung 870 EVO) | Windows 11 Home | H20: HydroLux PRO:HardLine Tubing| 1000w PSU

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

Hi Kilstorm,

The DLL goes in your panels folder, and the readme file included in the zip has a line to add to the panel.cfg file.

To my knowledge it hasn't been tested in P3D, please let me know if it works!

Farley

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

UPDATE ON UK ENGLISH ISSUE:

After several hours of research and pounding my head on the desk, I've reached the following conclusions:

1) It is not possible to use the copilot in Windows 7 with your Speech Recognition set to anything other than English(US)

2) It should be possible to use the copilot with the English(UK) setting in Windows 8 and later


This is due to a frustrating issue with Windows 7 and Microsoft Speech Recognition. The problem is this:

For an "In process" recognizer such as this, the SR engine REQUIRES that your SR language setting EXACTLY match your "Display Language" aka "MUI Language Pack". The problem is: THERE IS NO UK ENGLISH Language pack for Windows 7, or any other English pack for that matter. there is only "English" which is, in fact, US English. You can set your "input language" to English(UK), you can set your system locale to English(UK), but even if you set the queen's face as your desktop wallpaper, your "Display Language" will still be English(US), and unless your SR setting is English(US) it won't match, and the engine will not load. There is nothing I can do about it, nothing in my code requires US English, but according to all of my research and experimenting, Microsoft requires that these two settings match, and on Windows 7 this seems to be impossible unless SR is set to English(US).

Windows 8 and later appear to have a separate language pack for English(UK) which, if installed, would presumably eliminate this issue. If someone tries this, I would very much appreciate a post with the results.

For windows 7 users however, there is some comfort: It is quite easy to change your SR setting to English(US) when you want to use the copilot. ;) Just go to Control Panel\Ease of Access\Speech Recognition and click on Advanced Options

My sincere apologies for this issue, Lewis, I hope that you are still able to use and enjoy the copilot. Updates and support will continue. :D

Humbly,
Farley

P.S. How did it go with P3Dv3, Kilstorm?

User avatar
Kilstorm
Technical Sergeant
Posts: 555
Joined: 08 Sep 2009, 22:00

Re: Speech Recognition Copilot for the A2A B-17

Post by Kilstorm »

lequinne wrote:
P.S. How did it go with P3Dv3, Kilstorm?
I've read thru the PDF file a few times but dont see what I have to add to the panel.cfg file or where. Can you post what I need please?
i9-13900K O/C | ASUS ROG Maximus Z790 Hero | 32GB DDR5 6000MHz Kingston FURY | RTX 4090 24GB | 2x SSD M.2 (2TB Samsung 990 PRO) 1x SSD (4TB Samsung 870 EVO) | Windows 11 Home | H20: HydroLux PRO:HardLine Tubing| 1000w PSU

lequinne
Airman
Posts: 36
Joined: 06 Feb 2011, 16:25

Re: Speech Recognition Copilot for the A2A B-17

Post by lequinne »

Sorry Kilstorm,

I accidentally included a completely unrelated text file in my 1.1.0.zip, instead of the one with installation instructions!!! :oops: :oops: :oops:
That's embarrassing, I'm sorry!!

copied and pasted from the correct text file:

Installation follows normal FSX gauge protocol.
First place "B17CP.dll" in your "panel" folder here:

...\Microsoft Flight Simulator X\SimObjects\Airplanes\WoP2_b17g\panel

In this same folder you will find a file called "Panel.cfg"
In this file find the section near the end marked "[Vcockpit00]"

Add the following line at the bottom of this section:

gauge24=B17CP!B17_Copilot, 25,25,25,25


Very sorry about that, thanks for drawing my attention to it!!

Farley

User avatar
Kilstorm
Technical Sergeant
Posts: 555
Joined: 08 Sep 2009, 22:00

Re: Speech Recognition Copilot for the A2A B-17

Post by Kilstorm »

Things are looking good for P3Dv3!

First off, wow this is a great addon that you have created! I loaded it up late last night in P3Dv3 and I would say it works as planned but there is one small hiccup. When loading the B-17 on my system, the Control Panel is seen in a flash and then is gone and can not be brought up. I see the CoPilot in the addon menu bar but clicking on it does nothing. I do not have text across the top but so far that is not an issues as I get verbal replies that the command was heard. Guessing that is one of the items not checked in the checkbox.

That said, the voice commands work flawlessly and I love how the CoPilot uses the Mixture and Prop Locks (something I never did!). I also enjoy how you used the default sounds of "yes Sir", "Right away" etc as a confirmation that he received the command. I'm about to print out the list of commands and do a proper flight and will report back on how it goes. This is exactly what FS2Crew offers and is payware...you sir have a product that could be marketed and sold if you had a desire to trouble shoot/beta test the different platforms to get a working product in the different sims.

If you could do a screen grab of what the CP looks like so I could see the different options that would be great as I could assist in telling you what else I might be missing or is working, not working.

If there was a way to open the CP outside of the sim, that too might be an option.

I give this addon a 5 star rating as it is simple to install, highly useful and is immersive with the understanding that you are still testing it out.
i9-13900K O/C | ASUS ROG Maximus Z790 Hero | 32GB DDR5 6000MHz Kingston FURY | RTX 4090 24GB | 2x SSD M.2 (2TB Samsung 990 PRO) 1x SSD (4TB Samsung 870 EVO) | Windows 11 Home | H20: HydroLux PRO:HardLine Tubing| 1000w PSU

new reply

Return to “B-17 Flying Fortress”

Who is online

Users browsing this forum: No registered users and 10 guests