Web Application Development | Conference Calling App
At Spiral Scout we perform a lot of initial research for clients on the best possible solutions to a problem they are looking to solve. One recent client came to us with the challenge of creating a “Google Hangouts” between the US and Chinese universities. The general purpose of this research was to identify any potential solutions for a conferencing application between the US and China and to locate project specific issues during the early stages to predict any potential complexity in the future and where we might encounter issues. The research was performed in a standard matter where developers checked a list of available software and generated a set of recommendations and direction to take based on the best possible solutions in the marketplace.
Below is how we analyzed the problem and the solutions we came up with. If you are interested in building something similar please reach out to us at Spiral Scout or comment below.
Project-Specific Issues
This project has a few specific issues which were mainly linked to where the client was located.
Latency and Client Location
In order to minimize any buffering and debug connectivity issues between the US and China, we prepared a set of servers mainly located in Asia. For our initial tests, we picked two different options: Amazon EC2 servers (located in Tokyo) and DigitalOcean servers (located in Singapore).
Based on our research and the following tests there were no observed issues with connectivity or latency when communicating with servers directing towards the USA. We did observe a slight video and audio delay on this side of around 250–500ms which did not affect the call quality too significantly. However, a set of issues were observed when directing towards China. The team performed a set of connection tests with following results (IPS provider name were skipped in this table):
Region:Server:Test Observations: Chengdu, Sichuan Amazon EC2, Tokyo failed, all packets lost Nanjing, Jiangsu Amazon EC2, Tokyo100 ms ping, no packets lost Chongqing, Chongqing Amazon EC2, Tokyo100 ms ping, no packets lost Chengdu, Sichuan DigitalOcean, Singapore300–400 ms ping, package lost observed (25%) Nanjing, Jiangsu DigitalOcean, Singapore270–350 ms ping, no package loss Chongqing, Chongqing DigitalOcean, Singapore350–390 ms ping, no package lost
Given the above observations, this led us to believe that the average hosting provider may not have any extended integration into China’s intranet compared to Amazon’s network which exhibited the best results. In one particular test, we also observed a failed result linked to Chengdu, Sichuan — Amazon EC2 pair. We believe that this latency test may have failed due to local issues because in most cases everything worked fine on a non Amazon network (still some package loss was found). We recommended rerunning a ping on another date or from multiple computers to get a bigger sample size.
Please note: No Amazon servers in the Beijing region were tested. This allocation might show much better results for Chinese clients but will require connection and latency tests from USA side.
Connection and Quality Issues
eparately from a latency check, an additional set of issues were experienced related to video/audio quality and stability from the Chinese side. These tests were performed on a pre-selected FreeSwitch server located at Amazon EC2 (Tokyo).
From our understanding of the different video/audio conferencing software, most of the issues were related either to the client’s software, client’s bandwidth or client’s location. A deeper dive into this issue is listed in a section Maintenance and possible issues.
Note: each software that was tested required at least 1.0 MBit / 0.5 MBit bandwidth (in/out) or higher.
Great Firewall Issues
From our test, we did not observe any issues related to the Great Firewall besides some increased latency. Based on publicly available information, we can conclude that most of the issues can be caused by legal/political parts of this project, which would be beyond technical implementation
As you may know already, the client should be ready to act if their service is blocked from the Chinese side in case of any legal/political issues. We concluded that the choice of technology will affect it in any case, so as a rule of thumb, the rest of this research overview ignores any issues presented by the Great Firewall.
Communication Software Overview
This section gives a brief overview of technologies picked for this research, related issues and conclusions.
BigBlueButton provides pre-integrated solution for rich conference calling applications, including video and audio communication and whiteboard functionality.
Communication part of software based on RTMP protocol and following instruments:
- FreeSwitch — audio stream processing, SIP, WebRTC
- FFmpeg — format conversions
- Red5phone — video stream processing
- LibreOffice and ImageMagick — whiteboard support
- Nginx — interface delivery
- Flash — BBB interface implementation
- Redis — user actions capturing
API control
BigBlueButton can be controlled via a REST API and simplified for quick integration into a project.
Documentation can be found here.
Client Integration
Client integration involves loading a Flash based application, which handles parts like video/audio conferencing, whiteboard, and chat functionality.
An HTML5 client provides only audio and whiteboard support and is driven by a WebRTC protocol and FreeSwitch as a router. No video support is offered at this moment.
This means that the video is only available in desktop browsers that support Flash. No mobile browsers are supported in this case.
Issues: inability to share screen on modern browsers because there are no ActiveX plugins in a modern browser (Chrome, Firefox).
Mobile devices support
No mobile device support for video as there is no Flash.
Support
BigBlueButton provides both open source and commercial support via it authorized partners. The project is under active maintenance.
Development Considerations
BBB software is designed to provide quick integration into learning software, software presented as big solution package with finalized user flow and instruments set.
How to configure and install the Software is described in their documentation. However, if you want deeper integration and implementation of any specific flows you may encounter issues due to the set of technologies involved (Java, Redis, C, nginx configs, node.js and others).
We do not foresee the need for backend tweaking in the near future. However, a potential solution for any connection issues might involve server replication and traffic proxying, which can be complicated due to the size of the tech stack.
The largest integration considerations are related to the client side of the software which is written in Flash. Although this technology provides good support for legacy/old browsers, it’s used across the marketplace is rapidly declining due to modern HTML5 solutions (complexity and security issues: Flash vs WebRTC) and is not supported by mobile devices.
Conclusion
From our perspective, BBB seemed like a good candidate for an out-of-the box solution. However, any deeper integration and branding will require Flash developers with a vast knowledge base.
Since Flash is presented as separate application, we may find that seamless integration into the clients user flow and custom educational tools might not be even possible. Future browser and device support can also be treated as vote against this software.
From a development perspective, BBB will require too much abstraction and isolation which will require a lot of additional development as the project grows.
Apache OpenMeetings is another alternative to BigBlueButton since it provides similar functionality (excluding whiteboard) and is built using similar software.
This communication software is based on an RTMP protocol and driven by a same streaming server as in BBB — Red5. Their backend services are built using the Sprint framework (Java).
API Control
OpenMeetings provides a set of API endpoints that are available via REST and SOAP protocols and a set of pre-built plugins for different LMS applications.
Apache OpenMeeting received the least amount of research attention since it needed to function almost identically to BBB from backend perspective when only API and interfaces are different.
Client Integration
The client application is written in Flash and raised some of the same set of consideration.
Mobile devices support
No mobile device support for video as no Flash.
Support
OpenMeetings provides both open source and commercial support via its authorized partners.
Project is under active maintenance.
Development Considerations
Due high similarities between Apache OpenMeetings and BigBlueButton please refer to corresponding section in BBB above.
FreeSwitch is designed as a SIP and telephone software. However their latest, stable release includes support for audio and video communication and transcoding via WebRTC protocol.
This software is open-source and written primarily on C. It comes with set of modules for video transcoding (FFMpeg based), SIP telephony and bridging to other services (for example FreeSwitch serves as SIP and WebRTC audio support in BigBlueButton, see above).
API Control
Conference creation and conference controls can be performed by direct shell communication, REST api or via XMLRPC. Software also includes set of SDK libraries for different languages.
Client Integration
Due to the nature of WebRTC protocol, the client software can be written purely on JS and be seamlessly integrated into your final product.
You can find more details in WebRTC specific section.
Support
FreeSwitch provides both open source and commercial support via its authorized partners.
Project is under active maintenance.
Development Considerations
From a development perspective, FreeSwitch provides very low level abstraction of its services and modules which can simplify development by splitting architecture into smaller pieces but it can also add additional complexity because some pre-built modules may require client specific replacement (for example, chat and messaging can both be used inside a conference and outside of it).
FreeSwitch can be considered the simplest and smallest “integration unit” for the final software, which will require lowest level of abstraction and good flexibility. Since WebRTC is open-source and widely adopted protocol, the backend software (FreeSwitch itself) can be replaced to another package or SaaS solution without affecting the client software
Due to the SIP nature of FreeSwitch, it can also potentially integrate into the telephony network.
Conclusion
From our perspective, the FreeSwitch solution might be the best candidate for a long-standing project as it provides the best backend efficiency (see below), has a small amount of dependencies and has good separation between its client and server software.
Per this WIKI reference, FreeSwitch is already being used to power some commercial solutions.
Most of our conference testing was performed using FreeSwitch software.
Proposed Software
Based on our research, we proposed that the client use a WebRTC based solution combine with a backend written on FreeSwitch (or commercial analogue in a future). Our conclusion took into consideration a set of factors such as deeper in-product integration, high flexibility, potential scalability, lowest resource consumption and the separation of concerns between backend and client side of the application.
Our biggest focus was around the client side implementation and support, rather than backend infrastructure at this moment, as WebRTC provides us the ability to separate this side of the application and do development in parallel.
This section describes the basics and in-market WebRTC examples. The next section will focus on any frontend integration.
WebRTC Protocol Overview
WebRTC protocol defines a low level API that can be used to access a user’s device, input and implement real-time communication as client-server or even client-client. The biggest benefit of this protocol is that it provides native support in most modern browsers and mobile devices.
On the technical side, it works as a simple socket connection between client and recipient that does not require any specific hardware or network configuration. However, please note some issues and solutions linked to Amazon EC2 NAT configuration.
WebRTC is supported by multiple companies including Google, Firefox Foundation, W3C.
Current Market Examples
Currently, there are a number of commercial and open-source application which have built their client side using WebRTC:
WebRTC and Frontend Integration
As stated above, the client side (frontend) of the application is not directly linked to the backend implementation due to the open-source protocol in between. The following sections describe WebRTC specific integrations and possible concerns.
Browser Support
Although WebRTC is supported presented by most modern browsers, it is known that Android devices starting from 4.4.4, on IE9+ and OSX Safari browsers require an external plugin (confirmed working in IE9 and IE11).
Please note that the WebRTC protocol requires a valid SSL certificate in order to gain access to a user’s media device.
In our research, we checked a pre-built WebRTC implementation (Verto) which was supplied with FreeSwitch package.
Mobile Devices Support
Currently WebRTC only offers native browser support for Android devices. For Windows and IOS devices, an application would be required. Android 4.4.4+ tablets should work.
We additionally tested WebRTC support on Android MIUI 7+ devices and it performed well.
An application for non-Android devices may be implemented using a Cordova wrapper over a web based application. This will support WebRTC on mobile devices and could possibly provide the ability to utilize part of already written frontend code if we implement Cordova/Phonegap implementation.
An HTML5 based solution on WebRTC can be packed into a stand-alone cross-platform client and co-exist with a Web version.
This is achieved by packing the web version with web browsers. This approach has a few additional benefits:
- A user doesn’t need to install anything else (i.e. plugin for browser)
- Faster startup, as front-end code is packed within the application
- Additional functionality can be introduced that usually would not be possible with just a desktop browser, i.e. file system access for easier doc sharing and storage, screen sharing
One of technologies we’ve used for testing stand-alone software is Electron, which is a powerful actively supported tool that has a lot of products built into it.
Screensharing
Direct screen sharing may only be possible with a separate app (either just for screen sharing, either as a full featured stand-alone wrapper for everything, see section above).
Otherwise, support may be limited to Chrome and Firefox browsers with external plugins or extensions like https://github.com/muaz-khan/WebRTC-Experiment/tree/master/Pluginfree-Screen-Sharing
We have successfully tested and proved screen sharing using FreeSwitch software and Google Chrome plugin.
WebRTC based screen sharing can be demonstrated in commercial products such as uberconference.com, join.me and others.
Demo Test Analysis
Our initial demo tests were performed on a set of servers located in Asia. However our final tests were performed on one selected server located at Amazon EC2 Tokyo (server size was c4.xlarge). However after checking the load analysis, we believe the instance size can be reduced to lower CPU strain.
Tests were performed internally, in collaboration with the client and by the clients internal team only involving participants from USA, China and Europe. We chose Google Chrome for our tests in order to minimize effect of client and environment specific issues and focus on call quality and server configuration only. However the final product is intended to work on a wider set of browsers or via a downloadable application.
The following set of rooms were configured and tested for a purpose of this project (see next page)
Please note that server load is explained with relation to the average Linux load.
FreeSwitch, video muxing, room 3500
This room was configured in order to display every participant on a square screen equally without changing original video size.
Video codec: H.264 (15FPS)
Dynamic video transcoding: on
Audio: 16.000Hz, mono
Sound purification: on
This room configuration was unstable when the amount of participants increased to 6 people or above. The video quality we observed was ok and had issues with audio synchronization.
We observed multiple errors in the software and server logs and received audio failure when 8 people joined channel.
The server’s average load during the conference varied between 11.00 and 21.00 which is stated as very high.
Room configuration has been excluded from any future tests.
FreeSwitch, video muxing, room 3501 (internal tests only)
Room 3501 was configured similar to 3500 with a slight change made to audio and video codecs and quality (reduced bitrate rate, sound purification — off), we managed to reduce load to 12.00 for 12 participants which can still be treated as high.
Room configurations have been excluded from any future tests.
FreeSwitch, video mixing, speaker detection, room 3502
Channel were configured similarly to previously listed rooms with the addition of speaker detection. The person speaking was shown in a bigger tile than the participants not speaking. Video FPS was increased to 25 per second and video sizes were unified.
Video codec: H.264 (25FPS)
Dynamic video transcoding: on
Audio: 16.000Hz, mono
Sound purification: off
The tested room had better video quality than 3500 and 3501. However, we still noticed a slight audio desync between seeing lips move and hearing the words.
Speaker detection latency was recorded to be around 600ms.
The room had better stability when compared to any previous configurations. However the server load was 15.00–26.00 for 12 users and 2.00 до 8.00 for 4 users which can be labeled as moderate/high.
After we analyzed the log files, we can concluded that unifying the video size across clients can significantly reduce the CPU load. However it’s still high, when more users are involved.
Room configuration was used in in-field tests with low/moderate results on audio quality.
FreeSwitch, video switching, room 3504
The last conference room was configured differently so that video streams were not mixed together and only one users video stream was shown on a canvas (last speaker but not user itself).
User video streams were set to a pass-through mode meaning that no transcoding was performed on a server and FreeSwitch functioned as stream router.
Audio quality was set to 32.000Hz, stereo (purification on). We believe audio quality can be maximized to 44.000Hz without any side effects
Channels were tested on multiple rounds involving our internal development team, which included our testing and testing done only by the client.
During our initial and combined testing, this configuration showed the best quality for video and audio communication.
Speaker video stream was displayed on a full canvas size which can be seen as a benefit. Speaker detection latency was found to be around 600ms.
Because the rooms configuration excluded any video transcoding, the server was extremely low (0.01–0.10). In fact, it was close to a normally idle state.
The following configuration was shown to be the most promising results in terms of call quality and required processing resources and we recommend to use these configuration in your final product or future tests. Technical details related to 3504 room configuration are listed below, in the “Optimal configuration” section.
BigBlueButton test (does not relate to FreeSwitch, see above)
Separately from FreeSwitch, we performed a BigBlueButton software test on identical software. The client interface was driven by a Flash application. Servers were located in Asia on Amazon EC2 hardware.
Video and audio quality were noted as moderate. However, every participant was represented as a separate stream, which can be seen as a positive. There was low desynchronization between audio and video streams.
The observed server load was around 6.00–16.00 which can be described as moderate/high.
Client’s testing results
Separately from the combined tests, which only utilized one Chinese tester, there was a set of additional tests which were performed without involvement of SpiralScout team using various Chinese locations and regions. The results and conclusions can be seen below.
RegionObservations
Chengdu, Sichuan Failed:
- “Media Permission Denied”
Chongqing, Chongqing Failed:
- The test failed with the client
- The tester refreshed the page several times before seeing the meeting room where they need to dial the number.
- After Dialing the number 3504, the page stuck at “Determine Your Speed” for an extended period of time.
- Finally it went to the next page. At which moment, the timer was stuck at 00:00:00 forever.
- Couldn’t get in the meeting room.
Nanjing, Jiangsu Passed:
- No connection and quality issues, stable quality.
Panjin, Liaoning Passed with issues:
- The tester kept going back to the meeting room dial page after dialing 3504.
- After several attempts, the tester finally joined the meeting. The connection quality was ok, but not as good as the first testers.
- The tester exited the meeting room and tried to get back in. Then the first problem appeared again.
Since no errors related to initiating a conference were found on the server side, we believe that most of issues came from the client side. We will review some of them below:
Media Permission Denied
These issues highlight that Verto (the pre-built communication tool supplied with FreeSwitch) had an issue accessing the user hardware that can be caused by not configuring the SSL properly, certain issues with the frontend codebase or simply user error. We can’t certain what caused it because no client logs were accessible. However no other testers experienced these issues. Additional tests using the same user might help to identify the cause of this problem (We do not think this issue has any connection with the server setup or even internet stability).
Stability issues with regions Panjin, Liaoning and Chongqing, Chongqing
We experienced issues related to these tests. However these may be more serious and harder to solve. These issues are generally caused by having a non stable internet connection. Our assumption was based on the fact that the Verto and FreeSwitch server were analyzing the users connection speed by passing data packages with a known size. If the connection is interrupted, the speed test may get stuck in an undefined stage. Since Verto was written more like a demo tool for FreeSwitch, this maybe missing a set of code to automatically detect when the connection is lost and be able to handle it properly, which can cause a user to be thrown into the main dialing interface in order to join room instead of simply reopening the connection.
We recommended that the client perform additional tests for given testers with an enabled debug panel to capture all the browser console data. Also they could run the tests with the same users on the BigBlueButton server in the same location which might also help to identify if alternative solutions can handle the connection issues properly.
However, in order to solve this issue or to minimize it, a hard infrastructure update might be required. Please check the section “Connection Issues and potential solutions” for additional comments.
FreeSwitch Backend Integration
This section expanded on some of the technical details related to a FreeSwitch backend installation, configuration, existing/known issues and any potential solutions.
Server configuration and software required
For our initial deploy and testing, we tried two different approaches to install FreeSwitch using different lunix distributives.
The first tests were performed on Amazon EC2 instance with the following setup:
Ubuntu 14.04.5 LTS, Trusty, Intel(R) Atom(TM) CPU D525 4 Core, 2GM RAM
The following server configuration was chosen as one of cheapest and most common on a market However, the Ubuntu installation required a FreeSwitch compilation, which can be both as benefit (deeper configuration) and a negative (harder installation).
We are skipping the technical details for this installation as it required too much additional configuration, external packages and we didn’t foresee using it in production.
Installation showed good stability on this type of instance. However we did not manage to scale it properly for a bigger set of participants (Friday’s test failure).
Secondly, we have chosen one of the recommended FreeSwitch linux distributives and increased the instance size: Debian GNU/Linux 8, Xeon 4 Core, 8GM RAM
We did not providing details about the hard drive size as it was irrelevant to installation and configuration process.
Installation notes
Recommended installation notes can be based on this article with additional notes related to the production install:
- The following installation sequence configures the Verto client application. This section can be skipped with an application specific front-end
- Installation does not define the sequence for port mapping, NAT and Firewall configuration
- Installation uses an open-source solution for SSL generation. This part must be replaced with original certificates on the production build
In order to test the video recording, we enabled the following modules for the FreeSwitch platform for room configuration (some of modules are enabled by default):
autoload_configs/modules.conf.xml
<load module=”mod_console”/>
<load module=”mod_logfile”/>
<load module=”mod_enum”/>
<load module=”mod_xml_rpc”/>
<load module=”mod_cdr_csv”/>
<load module=”mod_event_socket”/>
<load module=”mod_sofia”/>
<load module=”mod_loopback”/>
<load module=”mod_rtc”/>
<load module=”mod_verto”/>
<load module=”mod_commands”/>
<load module=”mod_conference”/>
<load module=”mod_db”/>
<load module=”mod_dptools”/>
<load module=”mod_expr”/>
<load module=”mod_fifo”/>
<load module=”mod_hash”/>
<load module=”mod_voicemail”/>
<load module=”mod_esf”/>
<load module=”mod_fsv”/>
<load module=”mod_valet_parking”/>
<load module=”mod_httapi”/>
<load module=”mod_dialplan_xml”/>
<load module=”mod_dialplan_asterisk”/>
<load module=”mod_spandsp”/>
<load module=”mod_g723_1″/>
<load module=”mod_g729″/>
<load module=”mod_amr”/>
<load module=”mod_b64″/>
<load module=”mod_opus”/>
<load module=”mod_sndfile”/>
<load module=”mod_native_file”/>
<load module=”mod_png”/>
<load module=”mod_shout”/>
<load module=”mod_local_stream”/>
<load module=”mod_tone_stream”/>
<load module=”mod_lua”/>
<load module=”mod_say_en”/>
<load module=”mod_mp4v”/>
<load module=”mod_av”/>
In addition, all the rooms were using H.264 (compared to default H.263) codec which reduced the transcoding load by about 20% (however, this change did not affect room 3504 when there was no transcoding enabled).
At the bottom of the document, you can find materials we used to locate the most appropriate codec for video and audio (some resources have Russian origin which will require you to translate them first).
Amazon specific installation and NAT configuration
Separately from the software installation, we had to configure the server and Amazon firewall to pass traffic through the following ports:
udp 16384:32768
udp 4569
udp 5060
tcp 5060
udp 5080
tcp 5080
tcp 8000
udp 8000
Separately, note that the API communication over REST requires an additional port configuration. See below.
The port configuration must be done on a server via iptables and in Amazon Console -> EC2 -> Security groups.
Besides port mapping and configuration, we have to note that SIP and socket based software required an additional configuration in order to work behind NAT. We found that multiple references to proper configurations. However in our experiments only following definition worked:
autoload_configs/verto.conf.xml
<param name=”bind-local” value=”0.0.0.0:8081″/>
<param name=”bind-local” value=”0.0.0.0:8082″ secure=”true”/>
<param name=”rtp-ip” value=”{SERVER IP BEHIND NAT}”/>
<param name=”ext-rtp-ip” value=”{NAT EXTERNAL IP}”/>
You can find detailed information about the proper EC2 configuration referenced at bottom of this document. Please note that the first two lines of configuration were needed in order to make it work but not listed in the original documentation.
Optimal Configuration
As detected in room 3504, we noted that the best quality and resource consuming performance was here. We are attaching the configuration file compiled by for this channel:
autoload_configs/conference.conf.xml
<profile name=”video-35-room-test-4″>
<param name=”domain” value=”${domain}”/>
<param name=”rate” value=”32000″/>
<param name=”channels” value=”2″/>
<param name=”interval” value=”20″/>
<param name=”energy-level” value=”200″/>
<param name=”muted-sound” value=”conference/conf-muted.wav”/>
<param name=”unmuted-sound” value=”conference/conf-unmuted.wav”/>
<param name=”alone-sound” value=”conference/conf-alone.wav”/>
<param name=”enter-sound” value=”tone_stream://%(200,0,500,600,700)”/>
<param name=”exit-sound” value=”tone_stream://%(500,0,300,200,100,50,25)”/>
<param name=”kicked-sound” value=”conference/conf-kicked.wav”/>
<param name=”locked-sound” value=”conference/conf-locked.wav”/>
<param name=”is-locked-sound” value=”conference/conf-is-locked.wav”/>
<param name=”is-unlocked-sound” value=”conference/conf-is-unlocked.wav”/>
<param name=”pin-sound” value=”conference/conf-pin.wav”/>
<param name=”bad-pin-sound” value=”conference/conf-bad-pin.wav”/>
<param name=”caller-id-name” value=”${outbound_caller_name}”/>
<param name=”caller-id-number” value=”${outbound_caller_id}”/>
<param name=”comfort-noise” value=”false”/>
<param name=”conference-flags” value=”video-floor-only|rfc-4579|livearray-sync|minimize-video-encoding”/>
<param name=”video-mode” value=”passthrough”/>
<param name=”video-auto-floor-msec” value=”600″/>
<param name=”video-canvas-count” value=”1″/>
<param name=”video-super-canvas-label-layers” value=”true”/>
<param name=”video-super-canvas-show-all-layers” value=”true”/>
<param name=”video-canvas-size” value=”800×600″/>
<param name=”video-canvas-bgcolor” value=”#333333″/>
<param name=”video-layout-bgcolor” value=”#000000″/>
<param name=”video-codec-bandwidth” value=”1mb”/>
<param name=”video-fps” value=”30″/>
</profile>
As you can see, this room configuration provided the ability to specify every aspect of the channel behaviour, including colors, tone sounds, bandwidth limitations and etc.
You can find a link to the related document section attached to a bottom of this document.
API Control
As mentioned in the brief overview, the FreeSwitch room configuration and creation can be controlled in multiple ways. We recommend that to the client utilize XML-RPC protocol for this functionality which requires only a slight modification of the FreeSwitch configuration:
An additional module must be installed mod_xml_rpc and the following configuration section added:
autoload_configs/xml_rpc.conf.xml
<param name=”http-port” value=”8080″/>
<param name=”auth-user” value=””/>
<param name=”auth-pass” value=””/>
In order to gain access to the RPC API from the client side (not recommended for production), one must also open the designated port both in server Firewall and Amazon Security group.
Separately from the API based control, we recommended that the client could use the modules like mod_java and mod_python which let you directly communicate with the FreeSwitch server and access the raw stream data via a native server applications (we didn’t perform any further research on these modules).
Conference Recording
In our research, we also performed a test for the conference recording with positive results. However a few considerations are raised below. The following modules and configurations are required:
Modules: mod_mp4v, mod_vlc, mod_vpx (depends on the chosen recording format)
Channel configuration must be updated with the following section, which specifies the destination of the recording:
<param name=”auto-record” value=”/var/lib/freeswitch/recordings/${conference_name}_${strftime(%Y-%m-%d-%H-%M-%S)}.mp4″/>
VCL based recording requires a slightly different configuration. You can find the documentation here.
Please note that we tested the recording for rooms different than 3504 because video mixing is required (this is the opposite to 3504 where every client has their own video stream).
Possible solution might utilize the following approaches:
Note, audio recording were confirmed to be working in every room, including 3504 using the mod_conference specific configuration. You can find related documentation here.
We did not notice any load increase related to video recording (assuming that video stream was already encoded).
Room 3504 had video recording support as well. However it was undefined how the final conference video might appear when we recording was completed on a client to client basis. This issue can be addressed in development by selecting the appropriate format for a final video (side to side, speaker only and etc).
Additional notes and possible improvements
In addition to the listed configurations, we found a module which can provide Skype bridging for the calls — mod_skypopen. We have not performed any real tests using it. However you can find the related configuration notes in the following article (Russian source).
Connection issues and possible solutions
The biggest challenge for any selected software will be to ensure the proper connection quality between it recipients. In our tests, we did not notice any connection issues for a client located outside of China which leads us to solutions specific to the Chinese intranet.
After analyzing the traceroute information provided by the client, we can say that the latency issues are observed in relation to a following subnet 202.97.0.0/16:
inetnum: 202.97.0.0–202.97.31.255
netname: CHINANET-BB
descr: CHINANET backbone network
descr: Data Communication Division
descr: China Telecom
country: CN
admin-c: CH93-AP
tech-c: CH93-AP
mnt-by: APNIC-HM
status: ALLOCATED PORTABLE
source: APNIC
mnt-irt: IRT-CHINANET-CN
changed: hostmaster@ns.chinanet.cn.net 20000101
changed: hm-changed@apnic.net 20041214
A few tests also displayed addresses like 129.250.66.9 which are located in a middle of the traceroute dump and belong to USA subnets. We assume that VPN software was used.
Possible Solutions
Based on our assumptions and the traceroute results, we can see that most of connection issues are happening inside the Chinese intranet, meaning that the solution should focus on reducing the distance between Chinese users and the communications servers.
Quickest Approach
The quickest possible solution can be implemented by simply moving communication servers inside China. For example, utilizing Amazon Beijing EC2 region can drastically improve the user experience of Chinese users. Since Amazon networks showed the best possible ping-rate, moving servers to this region might not seriously affect United States users (must be double-checked).
Proxying and Bridging, Infrastructure
An alternative solution will require traffic bridging between China and USA servers. This approach might work well for both a set of users no matter where they are located. However it will require additional hardware and software configurations.
Proxying and Bridging, Software
On the software side (FreeSwitch), we can see a few different, potential implementations to manage traffic (in order of complexity):
- Bridging conference stream to another server.
- Managing each client with individual server (s1.each.com, s2.each.com, …), solution can also manage the server load by sharing video transcoding information between servers.
- Reverse engineer modules mod_verto.c, mod_conference.c in order to bypass video streaming to external service/server or even utilize Amazon Transcoding.
We believe that the connection issues with some Chinese regions may be solved on an infrastructure level rather than software level as the given problems might affect the selected conferencing implementation.
Feature set comparison
In order to unify research, we can provide you a table with quick overview of the feature set comparison between different software.
BigBlueButtonOpenMeetingsFreeSwitch
* We experienced issues with screen sharing using any Flash based software. In order for it to work, an ActiveX plugin was required (but forbidden in modern browsers).
** Via external plugin or native application.
*** The only challenge is to properly define how to glue videos from a multiple user streams. We believe this task is less technical but more design specific.
Conclusion
Based on this research, we recommended that the client use modern technologies like WebRTC and FreeSwitch rather than a Flash based solutions.
The chosen backend software demonstrated had good flexibility and rich functionality, when the front end segment of the application can be separated easily over the WebRTC protocol and provide a wide support for modern browsers as devices.
Legacy browsers can be supported using an external plugins or be completely replaced by a standalone client.
On a side note, integrating BigBlueButton solution might reduce the potential cost of development by utilizing most of the pre-integrated functions of this package, but we don’t think that the solution will provide a solid look for a user interfaces and might cause regression and legacy issues in the future which can lead to additional development down the road.
Quality and connection issues are mainly related to the Chinese intranet and will require additional development and infrastructure setup no matter what type of software is picked.
About Spiral Scout
Spiral Scout is a leading web design and development company based in San Francisco, CA. We are recognized as a top App Design & Development Company on DesignRush and a top web development company on Clutch. We specifically focus on developing digital asset management systems, enterprise solutions and e-commerce web applications.
About Spiral Scout
Spiral Scout is a leading web design and development company based in San Francisco, CA. In addition to being recognized as a Top App Design Company and a Top Web Development Company on DesignRush, Spiral Scout has also earned the honor of being named a Top Web Development Company on Clutch. We specifically focus on developing digital asset management systems, enterprise solutions and e-commerce web applications. Check out Spiral Scout on additional “best of” listings below:
Top Custom Website Design Companies
Top Small Business Web Design Companies
Top Mobile Web Design Companies
Top WordPress Web Design Companies
