Intelligent Ocular Image Generation

chevron-icon
Back
project-presentation-img
photrek
Project Owner

Intelligent Ocular Image Generation

Funding Awarded

$50,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 3.4 (5)

Status

  • Overall Status

    ⏳ Contract Pending

  • Funding Transfered

    $0 USD

  • Max Funding Amount

    $50,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$22,064 USD Pending TBD
Milestone Release 2
$17,648 USD Pending TBD
Milestone Release 3
$6,468 USD Pending TBD
Milestone Release 4
$3,820 USD Pending TBD

Project AI Services

No Service Available

Overview

AI applications developed for detection of medical conditions are hindered by lack of available data for training. Photrek contributes to the goals of DF4 with this Miscellaneous Pool proposal by generating synthetic image libraries for diseases of the human vision system and making these capabilities available to the community.
Photrek has built a team across three continents (Africa, Asia, North America) at four universities (U of Buea Cameroon, VIT India, KNUST Ghana, and SUNY Poly USA) with a team focused on image processing techniques applied to medical applications in a low cost environment within SNET goals for creation of democratic, decentralized, beneficial AGI.

Proposal Description

How Our Project Will Contribute To The Growth Of The Decentralized AI Platform

This project works to develop appropriate libraries and processes for synthetic images. Next steps include models offered on the SingularityNET Platform as APIs/services with usage-based pricing models for healthcare providers and researchers and sliding scale for researchers facing financial challenges. 

Potential future revenue streams include:

  1. Subscription fees from hospitals/clinics

  2. Pay-per-use model for AI model inference

  3. Custom model development/fine-tuning services

Our Team

We bring experienced, capable university faculty, development professionals, and managers with extensive background and capabilities in ML, AI, Statistics, and Scientific Computing together to develop algorithms and deliver a solid, robust, easy to use code base. All faculty have strong records of successful funded activity together with years of classroom experience at graduate and undergraduate levels, fluid communications skills, and administrative experience. This is a team set to deliver!

View Team

Company Name (if applicable)

Photrek https://www.photrek.io/home

The core problem we are aiming to solve

There is a critical lack of available data sets needed to train ML and AI solutions.

Applications are being developed to aid in the detection of diseases of the eye via fundus photographs and optical coherence tomography images [6]. Classifiers have been developed for conditions including detection of diabetic retinopathy [1], crossed eyes, bulging eyes, and conjunctivitis [2], macular oedema, age-related macular degeneration [3], loiasis and other parasites [4], cataracts, glaucoma, bulging eyes, and ocular hypertension [5]. Other approaches include eye disease classification using deep learning .

As these machine learning and artificial intelligence methods are developed to diagnose medical conditions, in particular those involving the human vision system, efforts are often hindered by the lack of available data sets on which to train algorithms [7] [8], [9]

We address this gap by developing methods for the synthetic generation of ocular images available to researchers in ML and AI and in particular to the SingularityNET community. 

Our specific solution to this problem

Photrek is keenly engaged in commercial grade data generation, with recent successes in image and video processing, and sees this as a natural area of application. 

This Deep Fund 4 New Projects Pool activity will 

  • leverage Photrek’s existing competencies in synthetic data generation and commercial hosting to develop a model for web hosting of AI and ML applications

  • work with the extensive professional networks of Photrek’s team to build a new research and development group across three continents, supporting talent development at four separate universities. We bring these diverse perspectives and expertise to bear on a problem of fundamental importance;

  • engage the SingularityNET community, through public forums such as the DeepFund TownHall and other avenues for feedback in order to ensure that the project is meeting community needs; and 

  • develop plans for future work and integrate this project with Photrek’s other SingularityNET efforts like Uncertain Knowledge Graphs by, for example, generating ocular images from text. This has proof of concept in other arenas such as Midjourney V5- text to image and  Da Vinci, though Foundational Models trained very broadly have difficulty producing the specific types of images needed.

Once a strong product is created and used fruitfully by the SNET community, we plan to work towards hosting on the SingularityNET AI Marketplace.

Project details

Our basic development processes will follow:

  1. Data Preparation: Curate and preprocess the ocular image datasets, ensuring proper labeling and annotation of disease severity levels.

 

  1. Training:

    1. Variational Auto-Encoder (VAE) [12]: Train a VAE model on the real ocular image data. The VAE will learn to encode the images into a latent space representation and generate new images from this latent space. We extending the basic VAE to a conditional VAE by conditioning the generation process on the desired disease severity level. This allows us to generate images with specified severity levels. We bring Photrel Risk Aware methodologies by using and assessing the Coupled VAE in this context. 

    2. Generative adversarial network (GAN) [7]: Implement the TrippleGAN architecture, which consists of three components: a generator, a discriminator, and a classifier. The generator will generate synthetic images, the discriminator will distinguish between real and generated images, and the classifier will classify the disease severity levels. Train the TrippleGAN in an adversarial manner, where the generator tries to produce realistic images that can fool the discriminator, while the discriminator tries to distinguish between real and generated images. The classifier will provide additional supervision by classifying the disease severity levels of both real and generated images.

  2. Evaluation: Once the generator is trained, we generate a large dataset of synthetic ocular images with varying disease severity levels and evaluate the performance of disease classification algorithms on this synthetic dataset, comparing the results to their performance on real data.

  3. Iterative Improvement: Use the evaluation results to identify weaknesses in the generated images or the classification algorithms. Refine the TrippleGAN architecture, training process, or classification models as needed, and repeat the evaluation process.

This approach leverages the power of VAEs for generating diverse and realistic ocular images, while the TrippleGAN architecture ensures that the generated images accurately represent the desired disease severity levels. The ability to create labeled synthetic data on-demand can be invaluable for training and testing ocular disease classification models, potentially improving their performance and robustness.

We anticipate following an ensemble strategy along the following lines. 

  1. Data Preprocessing and Augmentation:

    • Preprocess the ocular image data from various sources (e.g., fundus photographs, OCT scans, fluorescein angiography) to ensure consistency in image size, color space, and other relevant properties.

    • Perform data augmentation techniques like rotation, flipping, scaling, and brightness adjustments to increase the diversity of the training data and improve the model's generalization capability.

  2. Ensemble of Generative Models:

    • Train multiple TripleGAN models on different subsets of the ocular image data, each focusing on specific types of ocular images or disease conditions.

    • Employ different architectures, hyperparameters, and initialization strategies for each TripleGAN model in the ensemble to capture diverse representations and features.

  3. Ensemble of Discriminators and Classifiers:

    • For each TripleGAN model in the ensemble, train multiple discriminators and classifiers with different architectures, loss functions, and regularization techniques.

    • This diversity in the discriminators and classifiers can help capture various aspects of the image data, leading to more robust and accurate evaluation of the generated images.

  4. Ensemble Strategy:

    • During inference, generate images from each TripleGAN model in the ensemble, and combine the outputs using an appropriate ensemble strategy.

    • Possible ensemble strategies include averaging, majority voting, stacking, or more advanced techniques like mixture of experts or boosting.

  5. Ensemble Calibration and Refinement:

    • Evaluate the performance of the ensemble on a held-out validation set containing diverse ocular image types and disease conditions.

    • Use the evaluation results to calibrate the ensemble weights, retrain or fine-tune individual models, or adjust the ensemble strategy as needed.

    • Iteratively refine the ensemble until satisfactory performance is achieved across all types of ocular images.

  6. Continuous Learning and Adaptation:

    • Implement a continuous learning approach to update and adapt the ensemble as new ocular image data becomes available.

    • Periodically retrain or fine-tune individual models in the ensemble or introduce new models to maintain the ensemble's relevance and performance over time.

By employing an ensemble of TripleGAN models, discriminators, and classifiers, we leverage the strengths of different architectures and capture diverse representations of the ocular image data. This approach improves the generalization capability of the overall system, enabling it to work effectively on various types of ocular images while maintaining novelty and diversity in the generated samples.

Data Acquisition and Ethics

Obtaining ocular image data, especially involving patient information, requires careful handling from an ethical standpoint.  Researchers typically must partner with hospitals or clinics and obtain approval from ethics review boards to access de-identified/anonymized patient data for research purposes. 

For this phase of the project, we will use publicly available datasets like those from Kaggle competitions or research repositories, which have already gone through ethical vetting.

Exploring synthetic data generation approaches as in this proposal minimizes reliance on real patient data during development/testing. Regardless of the approach, maintaining data privacy, obtaining informed consent where applicable, and following ethical guidelines established by organizations like the World Medical Association will be crucial.

Technical Feasibility

For image processing, state-of-the-art deep learning models like convolutional neural networks (CNNs) and vision transformers will be employed. Specific architectures like EfficientNets, ResNets, etc. will be customized for the ocular disease classification task.

For generative modeling, techniques like Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and more recent approaches like Diffusion Models could be leveraged. These are established methods and we do not anticipate technical challenges beyond the implementation process.  

Cloud computing resources (TPUs/GPUs) will provide the computational power required to train these large AI models efficiently.

User Interaction Design

The user interface should prioritize ease of use for medical professionals. We will work towards a  simple upload interface for ocular images, with clear result visualization.

Advanced users could be provided with options to adjust parameters like desired disease severity levels. Interactive visualizations could allow users to explore the model's decision-making process.

Following accessibility guidelines and involving target users through user testing would be critical for an effective design.

Bibliography

[1] V. Gulshan et al., “Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs,” JAMA, vol. 316, no. 22, p. 2402, Dec. 2016, doi: 10.1001/jama.2016.17216.

[2] S. K. Sattigeri, “EYE DISEASE IDENTIFICATION USING DEEP LEARNING,” vol. 09, no. 07, 2022.

[3] D. S. W. Ting et al., “Artificial intelligence and deep learning in ophthalmology,” Br. J. Ophthalmol., vol. 103, no. 2, pp. 167–175, Feb. 2019, doi: 10.1136/bjophthalmol-2018-313173.

[4] S. Kumar, T. Arif, A. S. Alotaibi, M. B. Malik, and J. Manhas, “Advances Towards Automatic Detection and Classification of Parasites Microscopic Images Using Deep Convolutional Neural Network: Methods, Models and Research Directions,” Arch. Comput. Methods Eng., vol. 30, no. 3, pp. 2013–2039, Apr. 2023, doi: 10.1007/s11831-022-09858-w.

[5] A. A. Marouf, M. M. Mottalib, R. Alhajj, J. Rokne, and O. Jafarullah, “An Efficient Approach to Predict Eye Diseases from Symptoms Using Machine Learning and Ranker-Based Feature Selection Methods,” Bioengineering, vol. 10, no. 1, p. 25, Dec. 2022, doi: 10.3390/bioengineering10010025.

[6] T. Babaqi, M. Jaradat, A. E. Yildirim, S. H. Al-Nimer, and D. Won, “Eye Disease Classification Using Deep Learning Techniques.” arXiv, Jul. 19, 2023. Accessed: Mar. 27, 2024. [Online]. Available: http://arxiv.org/abs/2307.10501

[7] M. Fan, X. Peng, and X. Gong, “Ocular Disease Recognition and Classification using TripleGAN,” in Proceedings of the 2023 8th International Conference on Biomedical Signal and Image Processing, Chengdu China: ACM, Jul. 2023, pp. 7–11. doi: 10.1145/3613307.3613309.

[8] “Jin et al. - 2022 - FIVES A Fundus Image Dataset for Artificial Intel.pdf.”

[9] Z. Yu, Q. Xiang, J. Meng, C. Kou, Q. Ren, and Y. Lu, “Retinal image synthesis from multiple-landmarks input with generative adversarial networks,” Biomed. Eng. OnLine, vol. 18, no. 1, p. 62, Dec. 2019, doi: 10.1186/s12938-019-0682-x.

[10] “Shang et al. - 2024 - SynFundus-1M A High-quality Million-scale Synthet.pdf.”

[11] R. Nuzzi, G. Boscia, P. Marolo, and F. Ricardi, “The Impact of Artificial Intelligence and Deep Learning in Eye Diseases: A Review,” Front. Med., vol. 8, p. 710329, Aug. 2021, doi: 10.3389/fmed.2021.710329.

[12] D. P. Kingma and M. Welling, “An Introduction to Variational Autoencoders,” Found. Trends® Mach. Learn., vol. 12, no. 4, pp. 307–392, 2019, doi: 10.1561/2200000056.

Competition and USPs

The inspiration for this work stems partially from the participation of Photrek researchers Thistleton and Nelson in the National Science Foundation's Innovation Corps (I-Corps™). Compelling evidence gained from extensive conversations with leaders in industry and academia during the Customer Discovery process  impressed upon the Photrek team the lack of available data sets for development of new Machine Learning and Artificial Intelligence techniques across a variety of fields. 

It is difficult to find any commercial firms providing access to the “on demand” or "bespoke" synthetic images we propose to allow users to generate, so we believe Photrek and SingularityNET will be well positioned with this product.

Existing resources

We build this work on code and expertise developed for the DF3 Simulating a Risky World and code and expertise developed in Photrek’s Coupled-Variational AutoEncoder project. 

Proposal Video

DF Spotlight Day - DFR4 - William Thistelton - Intelligent Ocular Image Generation

4 June 2024
  • Total Milestones

    4

  • Total Budget

    $50,000 USD

  • Last Updated

    24 Jun 2024

Milestone 1 - Image Database Exploration and Needs Assessment

Status
😐 Not Started
Description

Objectives: Assess image libraries and incorporate several relevant libraries into our coding environment Preliminary image generation with VAE and Photrek cVAE Performance evaluation on synthetic image generation

Deliverables

Technical Report on design, implementation, and initial results

Budget

$22,064 USD

Link URL

Milestone 2 - Exploration of alternative generation processes

Status
😐 Not Started
Description

Objectives: Design details of use of probabilistic graph to form UKG Design details of use Draft architectural plan

Deliverables

Technical Report on design, implementation, and results.

Budget

$17,648 USD

Link URL

Milestone 3 - Initial Hosting and Community Feedback

Status
😐 Not Started
Description

Work with the community to develop an equitable model allowing under resourced researchers access to this product.

Deliverables

Report on alternative hosting schemes and community feedback.

Budget

$6,468 USD

Link URL

Milestone 4 - Final Report with Next Steps

Status
😐 Not Started
Description

Objectives: Respond to community feedback and document plans for next steps

Deliverables

Report on implementation of model and initial user results.

Budget

$3,820 USD

Link URL

Join the Discussion (2)

Sort by

2 Comments
  • 0
    commentator-avatar
    Gombilla
    Jun 2, 2024 | 5:10 PM

    Hello Photrek. I would like to comment on possible concerns associated with this project which may include the ethical implications of generating ocular images, particularly in terms of privacy and consent. There may also be challenges related to the accuracy and realism of the generated images, as well as ensuring that they are representative of diverse populations and ocular conditions. Are the approaches to put these considerations in check ?

    • 0
      commentator-avatar
      photrek
      Jun 18, 2024 | 3:16 PM

      Hi Gombilla- we'll be training on pubicly available data, at least initially. So, while we may need to work with one of our academic IRBs in the case of data collection, that's downstream from our immediate work. Regarding accuracy and realism, I think it's fair to say that this is one of the cornerstones of the project. We're looking at an ensemble approach which should help us on the creation side. We'll also, as you are pointing out, need to ensure that our metrics don't just measure realism in a generic and perhaps artificial way, but actually produce clinically authentic images. 

Reviews & Rating

New reviews and ratings are disabled for Awarded Projects

Sort by

5 ratings
  • 0
    user-icon
    Tu Nguyen
    Jun 6, 2024 | 7:47 AM

    Overall

    4

    • Feasibility 3
    • Viability 4
    • Desirabilty 3
    • Usefulness 4
    Intelligent Ocular Image Generation

    The problem this proposal raises is: There is a critical lack of available data sets needed to train ML and AI solutions. This proposal will create comprehensive image libraries of diseases of the human visual system and make these capabilities available to the community.
    I feel that they should define the start and end times of the milestones.

    user-icon
    photrek
    Jun 18, 2024 | 3:25 PM
    Project Owner

    Thanks, Tu Nguyen. We'll be doing the lion's share of this work (milestone 1 and 2) during summer 2024 (July/August). The less time intensive tasks will occur  early fall (Sept/Oct).

  • 0
    user-icon
    Joseph Gastoni
    May 20, 2024 | 4:02 PM

    Overall

    4

    • Feasibility 4
    • Viability 4
    • Desirabilty 4
    • Usefulness 5
    to generate synthetic image libraries

    This proposal outlines Photrek's project to generate synthetic image libraries for diseases of the human vision system. Here's a breakdown of its strengths and weaknesses:

    Feasibility:

    • High: The technical approach leverages established methods (VAEs, GANs) for image generation. Existing medical image datasets are available for training.
    • Strengths: The focus on building upon existing techniques and publicly available datasets reduces technical complexity.
    • Weaknesses: Fine-tuning the models for generating disease-specific images with varying severity levels might require significant computational resources.

    Viability:

    • Moderate: Success depends on the quality and usefulness of the synthetic images for training AI models in medical applications. Regulatory approval for using synthetic data in medical diagnosis might be needed.
    • Strengths: The focus on a critical need in medical AI development (data availability) addresses a current challenge.
    • Weaknesses: The proposal lacks details on the business model and how Photrek will ensure the quality and regulatory compliance of the synthetic data.

    Desirability:

    • High (potential): For AI developers in healthcare and researchers working on medical image analysis, this project can be desirable.
    • Strengths: The focus on addressing data scarcity in a specific medical domain (ophthalmology) aligns with a well-defined need.
    • Weaknesses: The proposal needs to clearly articulate the value proposition for researchers beyond just having more data.

    Usefulness:

    • High (potential): This project has the potential to significantly improve the development of AI-based tools for medical diagnosis.
    • Strengths: The focus on generating synthetic data for diseases of the human vision system addresses a critical bottleneck in developing AI for medical imaging.
    • Weaknesses: The proposal lacks details on how the project will measure the impact of the synthetic images on the performance of AI models in real-world applications.

    Additional Points:

    • Developing a clear strategy for user engagement and feedback collection from the medical AI community is crucial.
    • Establishing partnerships with medical institutions and AI developers can ensure the relevance and usefulness of the synthetic image libraries.
    • Addressing regulatory concerns regarding the use of synthetic data in medical diagnosis is important for long-term adoption.

    Overall, Photrek's project has the potential to be a valuable tool for advancing AI in healthcare. Focusing on data quality, regulatory compliance, user engagement, and partnerships can increase its effectiveness. By outlining a clear business model and impact measurement strategy, this proposal can become even more compelling.

    Here are some strengths of this project:

    • Addresses a critical bottleneck in medical AI development - lack of training data for specific diseases.
    • Leverages established generative modeling techniques (VAEs, GANs) to create synthetic images.
    • Focuses on a well-defined medical domain (ophthalmology) with a clear need for improved AI tools.

    Here are some challenges to address:

    • Ensuring the quality and realism of synthetic images to effectively train AI models.
    • Addressing regulatory hurdles around using synthetic data for medical diagnosis.

  • 0
    user-icon
    IlumNguyen
    Jun 6, 2024 | 3:58 AM

    Overall

    3

    • Feasibility 4
    • Viability 3
    • Desirabilty 3
    • Usefulness 3
    need a strategy to attract users

    During the Photrek project, I found that reliance on synthetic data can limit model accuracy and diversity, especially when there is not enough real-world data. I believe that, despite the project's emphasis on modeling and user acquisition, generating income from AI usage and development may also be difficult in terms of user acquisition and revenue. Managing a distributed team can also cause project management and coordination challenges. However, I also see many strengths in the project, including the use of advanced methods such as VAE and GAN models, the collaboration with universities and the expectation of integration into the market SingularityNET's AI. To be successful, I believe a project needs to address issues of feasibility, attractiveness, and profitability.

  • 0
    user-icon
    Max1524
    Jun 10, 2024 | 7:55 AM

    Overall

    3

    • Feasibility 3
    • Viability 3
    • Desirabilty 3
    • Usefulness 3
    Committed to quality assurance

    Photrek is the owner of many proposals in DFR4, so I need a commitment from the team to ensure the quality of this proposal, to avoid having to disperse personnel because of having to implement other proposals. Especially when the team has 4 anonymous members - why do they have to be anonymous?

    user-icon
    photrek
    Jun 11, 2024 | 2:50 PM
    Project Owner

    The anonymity is not intentional and is not necessary. Unfortunately, in implementing improvements to the proposal submission, it has also added some processing layers.  The Ocular Imaging project is an independent team of African and Indian scientists organized by Professor William Thistleton.  I will ask him to post in the comments the team biographies.  All the team members were invited to post their biographies; however, several of them are new to the ecosystem and unfamiliar with the process.

  • 0
    user-icon
    TrucTrixie
    Jun 9, 2024 | 3:28 PM

    Overall

    3

    • Feasibility 4
    • Viability 3
    • Desirabilty 3
    • Usefulness 3
    Is model accuracy and diversity limited?

    According to the author, what limits the accuracy and diversity of the model? Is it the dependence on aggregated data or is it due to some other reason? I hope the author makes this clear for the community to understand.

    Another suggestion to make the proposal better is that the team should completely disclose the identities of all members, not just 4/7 members as currently. Only when 7/7 members have their information publicly disclosed will the community have more trust in the team.

    user-icon
    photrek
    Jun 18, 2024 | 3:21 PM
    Project Owner

    Thanks, TrucTrixie- we're showning all the team members now. Please forgive us (some are academics) for getting caught up in the end of semester rush!

     

    We'll be training on several corpora of images and will try to understand the best way to aggregate our data on each of several generative tasks and procedures.

Summary

Overall Community

3.4

from 5 reviews
  • 5
    0
  • 4
    2
  • 3
    3
  • 2
    0
  • 1
    0

Feasibility

3.6

from 5 reviews

Viability

3.4

from 5 reviews

Desirabilty

3.2

from 5 reviews

Usefulness

3.6

from 5 reviews

Get Involved

Contribute your talents by joining your dream team and project. Visit the job board at Freelance DAO for opportunites today!

View Job Board